Research map / 3D vision / NeRF and radiance fields
ZeroRF: Fast Sparse View 360deg Reconstruction with Zero Pretraining
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ZeroRF, a per-scene sparse-view 360° NeRF method that parametrizes factorized (TensoRF-VM) feature grids with an untrained deep generator network (Deep Image Prior), requiring no pretraining or regularization.
Problem
Sparse-view 360° novel view synthesis with neural radiance fields: existing methods need heavy pretraining, handcrafted regularizers, or hours of per-scene optimization, and factorized grids overfit with noisy features under sparse views.
Approach
Integrate a tailored Deep Image Prior into factorized NeRF representations by generating TensoRF-VM planes and vectors from a randomly initialized convolutional generator fed with frozen Gaussian noise, so no pretraining or extra regularization is needed while training stays fast.
Key results
- On NeRF-Synthetic with 6 views, ZeroRF (full, ~25 min) reaches 24.73 PSNR / 0.889 SSIM / 0.113 LPIPS, versus 21.73 PSNR for vanilla TensoRF and 21.73 for FlipNeRF.
- After only 2 minutes (1k iters), ZeroRF reaches 21.42 PSNR at 4 views and 23.47 at 6 views on NeRF-Synthetic.
- On OpenIllumination, ZeroRF (1k iters, 2 min) reaches 26.26 PSNR, above DietNeRF (24.20) and InfoNeRF (22.28).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ZeroRF instead of FreeNeRF | Sparse-view reconstruction · NeRF-Synthetic, 4 views | PSNR: 18.81 -> 21.94 | Helped |
| Use ZeroRF instead of FreeNeRF | Sparse-view reconstruction · NeRF-Synthetic, 6 views | LPIPS: 0.149 -> 0.113 | Helped |
| Use ZeroRF instead of DietNeRF | Sparse-view reconstruction · OpenIllumination | PSNR: 24.20 -> 26.15 | Helped |
| Use ZeroRF instead of DietNeRF | Sparse-view reconstruction · OpenIllumination | SSIM: 0.940 -> 0.939 | Hurt |
| Add the ZeroRF prior to Triplane (ablation) | Triplane · NeRF-Synthetic, 6 views | PSNR: 20.55 -> 24.38 | Helped |
| Add the ZeroRF prior to DiF (ablation) | DiF · NeRF-Synthetic, 6 views | PSNR: 17.45 -> 23.56 | Helped |
| Add the ZeroRF prior to TensoRF (ablation) | TensoRF · NeRF-Synthetic, 6 views | PSNR: 21.73 -> 24.73 | Helped |
| Add a generator network (ablation) | TensoRF-VM ZeroRF · NeRF-Synthetic, 6 views | PSNR: 21.73 -> 24.73 | Helped |
| Replace the SD decoder generator with an MLP on grid (ablation) | TensoRF-VM ZeroRF · NeRF-Synthetic, 6 views | PSNR: 24.73 -> 20.69 | Hurt |
| Replace the SD decoder generator with SimMIM (ablation) | TensoRF-VM ZeroRF · NeRF-Synthetic, 6 views | PSNR: 24.73 -> 21.40 | Hurt |
| Replace the SD decoder generator with SD VAE (ablation) | TensoRF-VM ZeroRF · NeRF-Synthetic, 6 views | PSNR: 24.73 -> 24.68 | No clear effect |
| Replace the SD decoder generator with Kadinsky (ablation) | TensoRF-VM ZeroRF · NeRF-Synthetic, 6 views | PSNR: 24.73 -> 24.69 | No clear effect |
Surprising findings
- Handcrafted-prior methods such as FreeNeRF and FlipNeRF perform well on NeRF-Synthetic but fail catastrophically on OpenIllumination, indicating non-robust priors.
- Removing the frozen input noise (replacing it with a zero-initialized trainable feature) breaks the system completely, dropping PSNR to 12.21.
Limitations
- Extension to unbounded scenes is left as future work.
- Frozen noise input means the learned structure is fixed by the noise distribution; unfreezing gave no gains but added overhead.
Experimental setup
Datasets: NeRF-Synthetic OpenIllumination DTU
Benchmarks: NeRF-Synthetic OpenIllumination DTU
Metrics: PSNR SSIM LPIPS train time
Baselines: RegNeRF DietNeRF InfoNeRF FreeNeRF FlipNeRF TensoRF
Builds on: TensoRF Deep Image Prior SSDNeRF NeRF
Closest papers by meaning
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.89)
- MuRF: Multi-Baseline Radiance Fields (CVPR 2024 · similarity 0.89)
- FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors (CVPR 2025 · similarity 0.89)
- CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis (AAAI 2025 · similarity 0.89)
- Neural NeRF Compression (ICML 2024 · similarity 0.89)
- CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.