Research map / 3D vision / NeRF and radiance fields
NeRFPrior: Learning Neural Radiance Field as a Prior for Indoor Scene Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NeRFPrior, which overfits a grid-based NeRF (TensoRF) per scene as a density and color prior, then learns an SDF (NeuS-style) with occlusion-aware multi-view consistency and a confidence-weighted depth consistency loss for indoor surface reconstruction.
Problem
Indoor multi-view surface reconstruction with neural implicit functions suffers from weak priors: data-driven priors need large-scale pretraining and generalize poorly, while overfitting geometric priors lack color information and textureless regions are hard to reconstruct.
Approach
Use a per-scene trained NeRF (grid-based) as a prior supplying both density and color; query it as supervision for SDF-based volume rendering, use prior-based local volume rendering to check visibility for multi-view consistency, and add a confidence-weighted depth consistency loss for planar textureless regions.
Key results
- On ScanNet (4 scenes), the method without data-driven priors reaches F1 0.433 vs. 0.291 for NeuS and 0.292 for NeuralAngelo.
- On Replica, CD 0.038 and F1 0.813 vs. 0.066 CD and 0.665 F1 for NeuS.
- Total training pipeline time is 4.7h (37 min for the NeRF prior plus 4.2h training), about 50% less than prior neural implicit methods, per the paper's report.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add monocular cues | NeRFPrior · ScanNet | F1: 0.433 -> 0.782 | Helped |
| add ground-truth depth supervision | NeRFPrior · ScanNet | F1: 0.433 -> 0.930 | Helped |
| use NeRFPrior instead of NeuS | multi-view reconstruction · BlendSwap | F1: 0.483 -> 0.621 | Helped |
| use NeRFPrior instead of NeuS | multi-view reconstruction · Replica | F1: 0.665 -> 0.813 | Helped |
| add NeRF prior (ablation) | base implicit function network · Replica | F1: 0.483 -> 0.619 | Helped |
| add multi-view consistency (ablation) | base network with NeRF prior · Replica | F1: 0.619 -> 0.781 | Helped |
| add depth consistency loss (ablation) | NeRFPrior · Replica | F1: 0.744 -> 0.813 | Helped |
| remove Eikonal regularization (ablation) | NeRFPrior · Replica | F1: 0.794 -> 0.813 | Hurt |
Surprising findings
- Data-driven MonoSDF priors pretrained on indoor data degrade markedly on object-surrounding scenes (BlendSwap), as shown in Fig. 2.
Limitations
- Prior density field is noisy, requiring a threshold filter for supervision.
- Multi-view consistency and depth losses are only enabled late in training (after 100k and 150k iterations) because they can mislead early training.
Experimental setup
Datasets: ScanNet BlendSwap Replica
Benchmarks: ScanNet BlendSwap Replica
Metrics: Accuracy Completeness Precision Recall F1 Chamfer Distance Normal Consistency
Baselines: COLMAP NeRF UNISURF NeuS Geo-NeuS PermutoSDF NeuralAngelo Neural RGB-D Manhattan-SDF NeuRIS MonoSDF Go-Surf
Builds on: NeuS TensoRF MonoSDF Geo-NeuS UNISURF
Closest papers by meaning
- Dense Depth Priors for Neural Radiance Fields From Sparse Input Views (CVPR 2022 · similarity 0.92)
- NeRFVS: Neural Radiance Fields for Free View Synthesis via Geometry Scaffolds (CVPR 2023 · similarity 0.92)
- Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields (CVPR 2022 · similarity 0.91)
- Global and Hierarchical Geometry Consistency Priors for Few-shot NeRFs in Indoor Scenes (CVPR 2024 · similarity 0.91)
- Reducing Shape-Radiance Ambiguity in Radiance Fields with a Closed-Form Color Estimation Method (NeurIPS 2023 · similarity 0.91)
- Depth-Supervised NeRF: Fewer Views and Faster Training for Free (CVPR 2022 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.