Research map / 3D vision / NeRF and radiance fields
PaReNeRF: Toward Fast Large-scale Dynamic NeRF with Patch-based Reference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Speeds up SUDS-style large-scale dynamic NeRF for driving scenes by combining patch-based ray sampling, low-resolution rendering, and a lightweight CARN-based reference decoder that upsamples using patches retrieved via optical flow and structural similarity.
Problem
Large-scale dynamic NeRF reconstruction for autonomous driving suffers from very long training and rendering times, and low-resolution rendering with upsampling degrades image quality.
Approach
Replace random ray sampling with patch-based sampling, render a low-resolution feature map, and decode it with a reference decoder that fuses HR reference patches/images from known views, found by an optical-flow-restricted structural-similarity search.
Key results
- On KITTI image reconstruction, PaReNeRF reaches 32.64 PSNR and 0.933 SSIM versus SUDS 28.31 PSNR, cutting training time from 64 h to 26 h and inference from 12.86 s to 1.61 s.
- On KITTI novel view synthesis at 75% training views, PaReNeRF gets 25.19 PSNR versus SUDS 22.77.
- On VKITTI2 reconstruction, PaReNeRF reaches 30.894 PSNR versus reproduced SUDS 30.853, with inference 1.35 s versus 11.89 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PaReNeRF instead of SUDS | KITTI image reconstruction · KITTI image reconstruction | PSNR: 28.31 -> 32.642 | Helped |
| use PaReNeRF instead of SUDS | KITTI image reconstruction · KITTI image reconstruction | training time (h): 64 -> 26 | Helped |
| use PaReNeRF instead of SUDS | KITTI image reconstruction · KITTI image reconstruction | inference time (s): 12.86 -> 1.61 | Helped |
| use PaReNeRF instead of SUDS* | VKITTI2 image reconstruction · VKITTI2 image reconstruction | PSNR: 30.853 -> 30.894 | Helped |
| use PaReNeRF instead of SUDS | novel view synthesis · KITTI and VKITTI2, 75%, 50%, and 25% training views | PSNR: KITTI-75%: 22.77 -> 25.19; KITTI-50%: 23.12 -> 23.91; KITTI-25%: 20.76 -> 21.05; VKITTI2-75%: 23.87 -> 24.55; VKITTI2-50 | Mixed |
| add patch sampling (ablation) | PaReNeRF radiance fields · KITTI image reconstruction | PSNR: 28.31 -> 27.04 | Hurt |
| add patch sampling (ablation) | PaReNeRF radiance fields · KITTI image reconstruction | training time (h): 64 -> 36 | Helped |
| add encoder-decoder to patch sampling (ablation) | PaReNeRF radiance fields, without reference decoder · KITTI image reconstruction | inference time (s): 12.42 -> 1.75 | Helped |
| add encoder-decoder to patch sampling (ablation) | PaReNeRF radiance fields, without reference decoder · KITTI image reconstruction | PSNR: 27.04 -> 29.31 | Helped |
| reduce training iterations from 250K to 125K (ablation) | patch sampling + encoder-decoder, without reference decoder · KITTI image reconstruction | PSNR: 29.31 -> 28.65 | Hurt |
| add reference decoder (ablation) | patch sampling + encoder-decoder · KITTI image reconstruction | PSNR: 29.31 -> 33.23 | Helped |
| reduce training iterations from 250K to 125K (ablation) | full PaReNeRF · KITTI image reconstruction | PSNR: 33.23 -> 32.64 | Hurt |
Surprising findings
- Low-resolution rendering with CNN upsampling alone, reduced to 125K iterations, performed worse than the original SUDS in PSNR, so the reference decoder was needed to recover quality.
Limitations
- Baseline numbers for NeRF, NeRF+Time, NSG, PNF and SUDS on KITTI were taken from reported results, and SUDS on VKITTI2 was reproduced by the authors.
- Patch-based sampling alone reduces reconstruction accuracy, and low-resolution rendering alone needs longer training, so gains depend on combining components.
- Authors note that real-time training and rendering remain open challenges.
Experimental setup
Datasets: KITTI VKITTI2
Benchmarks: KITTI VKITTI2
Metrics: PSNR SSIM LPIPS training time inference time
Baselines: NeRF NeRF+Time NSG PNF SUDS UniSim
Builds on: SUDS NeRF CARN UniSim
Closest papers by meaning
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.89)
- CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis (AAAI 2025 · similarity 0.88)
- Optimize the Unseen - Fast NeRF Cleanup with Free Space Prior (NeurIPS 2025 · similarity 0.88)
- FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors (CVPR 2025 · similarity 0.88)
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.88)
- Ced-NeRF: A Compact and Efficient Method for Dynamic Neural Radiance Fields (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.