Research map / 3D vision / NeRF and radiance fields
ProvNeRF: Modeling per Point Provenance in NeRFs as a Stochastic Field
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Models per-point provenance (locations from which a 3D point is likely visible) as a stochastic field trained via a functional extension of IMLE, and uses it to improve sparse unconstrained NeRF novel view synthesis and triangulation uncertainty estimation.
Problem
Existing NeRFs do not explicitly model the distribution of training camera poses or triangulation quality, which strongly affects reconstruction quality in sparse, unconstrained view settings.
Approach
Represent each point's provenance as samples from a learned distribution, modeled as a stochastic field by extending implicit maximum likelihood estimation (IMLE) to functional space (fIMLE), with a derived pointwise loss that is efficient for gradient descent; the provenance field is attached to a pretrained NeRF and used both as a regularizer for NVS and to compute triangulation uncertainty.
Key results
- On ScanNet NVS, ProvNeRF reaches 21.73 PSNR / 0.733 SSIM / 0.291 LPIPS versus 21.54 PSNR for SCADE; on Tanks and Temples it reaches 20.36 PSNR versus 20.13 for SCADE.
- Triangulation uncertainty NLL on ScanNet averages -3.05 versus 5.47 for the best baseline Bayes' Rays, and on Matterport3D averages -11.0 versus 5.49.
- Preliminary 3D Gaussian Splatting extension reports lower NLL than FishRF on three ScanNet scenes, with post-training of about 30 minutes on one A6000 GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add per-point provenance modeling and provenance NVS regularization | SCADE · ScanNet | PSNR: 21.54 -> 21.73 | Helped |
| Add per-point provenance modeling and provenance NVS regularization | SCADE · Tanks and Temple | LPIPS: 0.358 -> 0.349 | Helped |
| Use provenance-based triangulation uncertainty | SCADE / DDP · ScanNet Avg. and Matterport Avg. | NLL: 7.71 -> -3.05; 63.0 -> -11.0 | Helped |
| Freeze the provenance branch rather than jointly updating it with the NeRF (ablation) | ProvNeRF NVS · ScanNet | PSNR: 21.73 -> 21.68 | Hurt |
| Use a spatially invariant latent stochastic field (ablation) | ProvNeRF · ScanNet | PSNR: 21.73 -> 21.63 | Hurt |
| Replace fIMLE with a Gaussian-based field, C=2 (ablation) | ProvNeRF · ScanNet | PSNR: 21.73 -> 20.77 | Hurt |
| Replace fIMLE with a Gaussian-based field, C=5 (ablation) | ProvNeRF · ScanNet | PSNR: 21.73 -> 21.04 | Hurt |
| Replace fIMLE with a VAE-based model (ablation) | ProvNeRF · ScanNet | PSNR: 21.73 -> 19.28 | Hurt |
| Increase the number of sampled random functions K (ablation) | Provenance field · ScanNet | AP: 0.197 -> 0.534 -> 0.644 -> 0.738 -> 0.745 | Helped |
Surprising findings
- A deterministic provenance field and frustum-check baselines underperform the stochastic fIMLE field, and AUSE is argued to be an unreliable uncertainty metric in some cases.
Limitations
- Requires post-hoc optimization of about 8 hours, limiting real-time or on-demand use
- Hyperparameters are tuned per application for best performance rather than adaptively
- Gains on NVS over SCADE are modest (e.g., 0.2 dB PSNR on ScanNet)
Experimental setup
Datasets: ScanNet Matterport3D Tanks and Temples
Benchmarks: ScanNet Tanks and Temples Matterport3D
Metrics: PSNR SSIM LPIPS negative log-likelihood average precision AUC
Baselines: NeRF DDP SCADE DäRF CF-NeRF Bayes' Rays Ensemble FisherRF
Builds on: implicit maximum likelihood estimation NeRF SCADE DDP
Closest papers by meaning
- Optimize the Unseen - Fast NeRF Cleanup with Free Space Prior (NeurIPS 2025 · similarity 0.89)
- FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors (CVPR 2025 · similarity 0.89)
- ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization (CVPR 2024 · similarity 0.89)
- CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning (AAAI 2024 · similarity 0.89)
- Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video (CVPR 2025 · similarity 0.89)
- NeRFPrior: Learning Neural Radiance Field as a Prior for Indoor Scene Reconstruction (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.