Research map / 3D vision / NeRF and radiance fields
Optimize the Unseen - Fast NeRF Cleanup with Free Space Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a post-hoc NeRF cleanup that adds a sigmoid-softened Free Space Prior loss on density at uniformly sampled 3D points (not just along rays) to remove floaters in unseen regions, fine-tuning the existing NeRF with no extra memory.
Problem
NeRF photometric optimization leaves density in unseen regions unconstrained, producing floaters that degrade novel views; existing cleanup methods (Nerfbusters, BayesRays) are costly in cleanup time or inference time.
Approach
Apply a global zero-density prior via a sigmoid-softened loss on densities at randomly sampled points across the entire 3D space (including unseen and occluded regions), balanced against the photometric loss along training rays, framed as MAP estimation rather than maximum likelihood.
Key results
- Free Space Prior cleanup trains in 25 s versus 37 s for BayesRays and 1200 s for Nerfbusters, and its inference is 205 ms/frame versus 506 ms for BayesRays (2.5x faster).
- At matched coverage, Ours (lambda=0.01) and BayesRays (T=0.9) give comparable PSNR; Ours at lambda=10^0 reaches PSNR 17.69 with coverage 87.57% on Nerfbusters, versus Nerfbusters at PSNR 17.01 and coverage 68.13%.
- Sparse-view experiment on Garden (24 images) gives SSIM 0.640 and LPIPS 0.257 for Ours versus 0.637 and 0.261 for the base model, with comparable PSNR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a 3D Free Space Prior during post-hoc cleanup | Nerfacto · Nerfbusters | PSNR: 16.44 -> 17.60 | Helped |
| add a 3D Free Space Prior during post-hoc cleanup | Nerfacto · Nerfbusters | Coverage (%): 98.48 -> 89.88 | Hurt |
| use Free Space Prior instead of BayesRays | NeRF cleanup · Nerfbusters | Cleanup Time (s): 37 -> 25 | Helped |
| use Free Space Prior instead of BayesRays | NeRF rendering · Nerfbusters | Inference Time (ms/frame): 506 -> 205 | Helped |
| use full 3D sampling rather than sampling the prior only along training rays (ablation) | Free Space Prior · NeRF cleanup | artifact removal | Helped |
| increase randomly sampled 3D points to N=2^{17} (ablation) | Free Space Prior · Pikachu | cleanup quality: 10% | Helped |
| vary λ across the reported range (ablation) | Free Space Prior · Nerfbusters | PSNR: 17.35–17.69 | No clear effect |
| apply the Free Space Prior during training rather than post-hoc (ablation) | NeRF training · Nerfbusters | PSNR: 17.30 | Mixed |
| use the Free Space Prior on sparse training views (ablation) | Garden scene · MipNeRF-360 | SSIM: 0.637 -> 0.640; 0.703 -> 0.705; 0.724 -> 0.727 | Helped |
Surprising findings
- The prior is also applied behind surfaces and occluded regions, and it reduces density in visible regions as a side effect, which the authors balance against the photometric loss.
- On underwater Seathru scenes the density histogram does not become bimodal, because attenuation and backscatter mean density should not be zero; the prior still helps there.
Limitations
- Cannot correct artifacts caused by image inconsistencies, moving objects, poor bundle adjustment, or noisy input images; the noisy-image study shows color bias remains.
- The zero-density assumption does not hold in media like underwater scenes, where density should be non-zero.
- Lambda trades off coverage against PSNR, and the stricter predicted-mask evaluation is the authors' own modification of the protocol.
Experimental setup
Datasets: Nerfbusters Mip-NeRF 360 Light Fields Tanks and Temples PhotoTourism Seathru-NeRF KITTI-360 LeRF
Benchmarks: Nerfbusters
Metrics: PSNR SSIM LPIPS Coverage Dice Chamfer Distance Depth RMSE Depth MAE
Baselines: Nerfbusters BayesRays Gaussian Splatting L2 regularization Nerfacto
Builds on: NeRF Nerfacto Nerfbusters BayesRays RegNeRF
Closest papers by meaning
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.91)
- Removing Objects From Neural Radiance Fields (CVPR 2023 · similarity 0.91)
- Aug-NeRF: Training Stronger Neural Radiance Fields With Triple-Level Physically-Grounded Augmentations (CVPR 2022 · similarity 0.90)
- ProvNeRF: Modeling per Point Provenance in NeRFs as a Stochastic Field (NeurIPS 2024 · similarity 0.89)
- LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes (NeurIPS 2024 · similarity 0.89)
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.