Research map / 3D vision / Gaussian splatting
NerfBaselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NerfBaselines, a framework that wraps official NeRF and 3DGS codebases in isolated environments under a unified evaluation protocol, showing that small protocol differences change method rankings and releasing a web benchmark.
Problem
Novel view synthesis (NeRF and 3DGS) comparisons are unreliable because methods use different evaluation protocols (downscaling, background color, LPIPS variant, metric details), codebases are hard to install, and reported numbers are hard to reproduce.
Approach
Rather than reimplementing methods, wrap the official code behind a common method API (train_iteration, render, etc.), freeze dependencies in isolated Docker/Apptainer/Conda environments, and standardize a single evaluation protocol per dataset, then quantify how protocol changes alter rankings.
Key results
- On Mip-NeRF 360, switching from official downscaling to manual downscaling changes 3DGS-based rankings; Gsplat moves from rank 6 to rank 5 under P2 in PSNR across protocols.
- On Blender, using a black instead of white background boosts Mip-Splatting from rank 2 to rank 1 (PSNR 33.85 vs 33.33 under P1).
- NerfBaselines reproduces most published numbers within about 1% PSNR, e.g. Zip-NeRF 28.553 vs 28.54 paper on Mip-NeRF 360.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Manually downscale the original large images instead of using released downscaled images | Mip-NeRF 360 methods · Mip-NeRF 360 | PSNR: Zip-NeRF: 28.55$\pm$0.01 -> 28.84$\pm$0.01; Scaffold-GS: 27.71$\pm$0.01 -> 28.01$\pm$0.02 | Mixed |
| Use a black instead of white blending background | Blender methods · Blender | PSNR: GOF: 33.45 -> 33.76; Mip-Splatting: 33.33 -> 33.85 | Helped |
| Optimize appearance embeddings on full test images instead of the left half | Photo Tourism methods · Photo Tourism | PSNR: WildGaussians: 24.56 -> 25.96 | Helped |
| Round predictions to the uint8 range during evaluation | NerfBaselines evaluation · Novel view synthesis evaluation | PSNR: approximately $\pm0.02$ PSNR | Mixed |
Surprising findings
- Tiny evaluation protocol differences (downscaling method, background color) can change method rankings more than the gap between methods themselves.
- NerfStudio re-implementations of NeRF methods underperform the originals (e.g., Instant-NGP lego 15.24 vs 36.39 PSNR in NerfStudio).
Limitations
- Adoption by the research community is uncertain; the framework's value depends on continued maintenance.
- Some methods could not be reproduced exactly due to codebase evolution or unreleased hyperparameters (e.g., Tetra-NeRF, Instant-NGP on Blender).
Experimental setup
Datasets: Mip-NeRF 360 Blender Tanks and Temples Photo Tourism LLFF SeaThru-NeRF NeRF On-the-go
Benchmarks: Mip-NeRF 360 Blender Tanks and Temples Photo Tourism
Metrics: PSNR SSIM LPIPS training time GPU memory
Baselines: NerfStudio Instant-NGP Zip-NeRF Mip-NeRF 360 3DGS Mip-Splatting Scaffold-GS Gaussian Opacity Fields gsplat 2DGS
Builds on: NerfStudio Instant-NGP 3D Gaussian Splatting Mip-NeRF 360 viser
Closest papers by meaning
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.90)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.89)
- A General Implicit Framework for Fast NeRF Composition and Rendering (AAAI 2024 · similarity 0.88)
- NeRF Director: Revisiting View Selection in Neural Volume Rendering (CVPR 2024 · similarity 0.88)
- ProvNeRF: Modeling per Point Provenance in NeRFs as a Stochastic Field (NeurIPS 2024 · similarity 0.88)
- Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.