Research map / 3D vision / Depth and stereo
Neural Spline Fields for Burst Image Fusion and Layer Separation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-layer alpha-composited neural spline field model that jointly fuses a handheld long-burst capture into a high-resolution image and separates transmission and obstruction layers via test-time optimization.
Problem
Burst image fusion collapses scene motion, parallax, and occlusions into a single image, while decomposing these effects from misaligned burst frames is highly ill-conditioned; the goal is joint reconstruction and layer separation for occlusion removal, reflection suppression, and shadow removal.
Approach
Models each burst frame as alpha compositing of transmission and obstruction planes with a projective camera model, where per-layer flow is produced by neural spline fields (coordinate networks outputting cubic Hermite spline control points), giving temporally smooth flow without regularization.
Key results
- On synthetic occlusion scenes the proposed method reaches about 20.82/0.694 PSNR/SSIM versus 19.58/0.625 for the best prior baseline shown in Figure 8 (per the figure's scene), and 33.84/0.985 on the reflection example versus at most 21.86/0.829 for baselines.
- Fits a 42-frame sequence in about 3 minutes on a single RTX 4090 GPU with no post-processing or learned priors.
- Qualitatively outperforms Lama, Liu et al., OCC-NeRF, NIR, NeRFReN and DSR-Net on real in-the-wild occlusion and reflection captures.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method instead of Lama | Synthetic occlusion scenes · Synthetic occlusion scenes | PSNR: 20.82 -> 41.20 | Helped |
| Use the proposed method instead of Lama | Synthetic occlusion scenes · Synthetic occlusion scenes | SSIM: 0.694 -> 0.982 | Helped |
| Use the proposed method instead of NIR | Synthetic reflection scenes · Synthetic reflection scenes | PSNR: 21.86 -> 33.84 | Helped |
| Use the proposed method instead of NIR | Synthetic reflection scenes · Synthetic reflection scenes | SSIM: 0.829 -> 0.985 | Helped |
Surprising findings
- Multi-view methods fail to achieve meaningful occlusion removal on small-baseline burst captures because the occluder maintains high self-overlap across the sequence.
- Pretrained segmentation (SAM) and optical flow (RAFT) models fail on small-motion scenes with out-of-focus occluders.
Limitations
- Requires long burst captures with camera metadata and gyroscope data, and per-scene test-time optimization.
- Pretrained flow and segmentation models do not transfer to small-motion data with large obstructions.
Experimental setup
Datasets: in-the-wild long-burst captures (42 frames, 12MP RAW, Pixel 7/7-Pro/8-Pro) synthetic rendered scenes with paired ground truth using measured hand-shake motion [9]
Benchmarks: synthetic rendered obstruction removal scenes with paired ground truth
Metrics: PSNR SSIM
Baselines: Lama Liu et al. multi-view obstruction removal OCC-NeRF NIR NeRFReN DSR-Net SAM RAFT
Builds on: NeRF-based layered neural image model (NIR) Long-burst photography (Chugunov et al.) multi-resolution hash encoding (Instant-NGP) relative photometric loss RAFT
Closest papers by meaning
- TurboSL: Dense Accurate and Fast 3D by Neural Inverse Structured Light (CVPR 2024 · similarity 0.85)
- Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.84)
- Splatter Image: Ultra-Fast Single-View 3D Reconstruction (CVPR 2024 · similarity 0.84)
- Robust Neural Rendering in the Wild with Asymmetric Dual 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.84)
- Gated Fields: Learning Scene Reconstruction from Gated Videos (CVPR 2024 · similarity 0.83)
- Horizon-GS: Unified 3D Gaussian Splatting for Large-Scale Aerial-to-Ground Scenes (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.