Research map / Point clouds / Point cloud classification
Depth-Guided Robust and Fast Point Cloud Fusion NeRF for Sparse Input Views
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a depth-guided NeRF for sparse views that builds per-view point clouds encoded in TensoRF-style vector-matrix factors and fuses them into a voxel grid, reducing sensitivity to inaccurate depth and speeding reconstruction.
Problem
Novel-view synthesis from sparse input views with NeRF suffers from overfitting; existing depth-guided methods ignore depth map inaccuracies and have low time efficiency.
Approach
Each input view's RGB-D point cloud is encoded as a set of vector-matrix factors (one component per view) in a TensoRF-style feature grid; the factors of all views are summed to form a fused scene point cloud, so inaccurate depths are refined or replaced by other views during training.
Key results
- On LLFF with 2 views, the method reaches 17.83 PSNR / 0.5512 SSIM / 0.3832 LPIPS, versus 17.21 PSNR for the best baseline DDP-NeRF.
- With 4 views on LLFF, PSNR is 20.86 versus 20.11 for DSNeRF.
- Adding 10% noise to COLMAP depths lowers LLFF PSNR from 17.83 to 13.26, showing sensitivity to depth quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use point cloud construction and fusion instead of random matrices and vectors (ablation) | Our Method · LLFF | quantitative outcomes | Helped |
| add 5% white noise to COLMAP depth maps (ablation) | Our Method · LLFF | PSNR: 17.83 -> 15.98 | Hurt |
| add 10% white noise to COLMAP depth maps (ablation) | Our Method · LLFF | PSNR: 17.83 -> 13.26 | Hurt |
| add 5% white noise to COLMAP depth maps (ablation) | Our Method · DTU | PSNR: 19.34 -> 18.17 | Hurt |
| add 10% white noise to COLMAP depth maps (ablation) | Our Method · DTU | PSNR: 19.34 -> 16.63 | Hurt |
| use point cloud fusion NeRF instead of DDP-NeRF | LLFF (2 views) · LLFF (2 views) | PSNR: 17.21 -> 17.83 | Helped |
| use point cloud fusion NeRF instead of DDP-NeRF | LLFF (2 views) · LLFF (2 views) | SSIM: 0.5377 -> 0.5512 | Helped |
| use point cloud fusion NeRF instead of ViP-NeRF | LLFF (2 views) · LLFF (2 views) | LPIPS: 0.4017 -> 0.3832 | Helped |
| use point cloud fusion NeRF instead of DSNeRF | LLFF (3 views) · LLFF (3 views) | PSNR: 19.02 -> 19.30 | Helped |
| use point cloud fusion NeRF instead of ViP-NeRF | LLFF (3 views) · LLFF (3 views) | SSIM: 0.5837 -> 0.6027 | Helped |
| use point cloud fusion NeRF instead of ViP-NeRF | LLFF (3 views) · LLFF (3 views) | LPIPS: 0.3750 -> 0.3682 | Helped |
| use point cloud fusion NeRF instead of DSNeRF | LLFF (4 views) · LLFF (4 views) | PSNR: 20.11 -> 20.86 | Helped |
Surprising findings
- Under 10% depth noise, LLFF PSNR drops by about 4.6 dB, which contradicts the claim of robustness to inaccurate depth.
Limitations
- Depth-quality robustness is limited: performance degrades substantially with 10% noisy depth.
- Relies on COLMAP-generated depth maps and camera poses.
- Paper's limitations section is only a brief statement about future work.
Experimental setup
Datasets: LLFF DTU
Benchmarks: LLFF DTU
Metrics: PSNR SSIM LPIPS
Baselines: InfoNeRF DietNeRF RegNeRF DSNeRF DDP-NeRF ViP-NeRF
Builds on: TensoRF DSNeRF NeRF
Closest papers by meaning
- NexusGS: Sparse View Synthesis with Epipolar Depth Priors in 3D Gaussian Splatting (CVPR 2025 · similarity 0.90)
- Point2Pix: Photo-Realistic Point Cloud Rendering via Neural Radiance Fields (CVPR 2023 · similarity 0.89)
- Learning Non-Volumetric Depth Fusion Using Successive Reprojections (CVPR 2019 · similarity 0.88)
- Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection (AAAI 2024 · similarity 0.86)
- Sparse Views Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo (CVPR 2024 · similarity 0.86)
- 3DMambaIPF: A State Space Model for Iterative Point Cloud Filtering via Differentiable Rendering (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.