Research map / 3D vision / NeRF and radiance fields
DetRF: Detachable Novel Views Synthesis of Dynamic Scenes Using Backdrop-Driven Neural Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DetRF, a parallel dual-NeRF with a backdrop-driven static background encoder and a per-ray occlusion/transmittance weight module, for novel view synthesis of dynamic scenes from casual monocular videos.
Problem
Novel view synthesis of real-world dynamic scenes from casual monocular video is ill-posed; prior NeRF methods ignore the global static background distribution and occlusion transmittance along rays, hurting static and occluded regions.
Approach
Decompose the scene into a static background NeRF conditioned on a latent backdrop code (via attention over a backdrop encoder) and a 6D-input dynamic NeRF with flow prediction, then blend their renderings per ray using learned per-sample occlusion weights and a transmittance weight.
Key results
- On NVIDIA dynamic scenes, DetRF reaches 27.55 PSNR and 0.0530 LPIPS on average, versus 27.30/0.0603 for Gao et al. (2021).
- On urban driving scenes, DetRF reaches 33.16 PSNR and 0.9586 SSIM, versus 32.85 PSNR for Gao et al.
- Ablation: the backdrop encoder adds 0.79 PSNR and the occlusion weight module adds about 5.5% PSNR on NVIDIA scenes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DetRF instead of Gao et al. | NVIDIA dynamic scenes · NVIDIA dynamic scenes (Ave) | PSNR: 27.30 -> 27.55 | Helped |
| Use DetRF instead of Gao et al. | NVIDIA dynamic scenes · NVIDIA dynamic scenes (Ave) | LPIPS: .0603 -> .0530 | Helped |
| Use DetRF instead of Gao et al. | Urban driving scenes · Urban driving scenes | PSNR: 32.85 -> 33.16 | Helped |
| Use DetRF instead of Gao et al. | Urban driving scenes · Urban driving scenes | SSIM: .9571 -> .9586 | Helped |
| Use DetRF instead of NSFF | Urban driving scenes · Urban driving scenes | LPIPS: .0413 -> .0359 | Helped |
| Add backdrop encoder (BE) (ablation) | Base (two naive NeRFs) · NVIDIA dynamic scenes | PSNR: 25.23 -> 26.02 | Helped |
| Add ray occlusion weight module (RW) (ablation) | Base+BE · NVIDIA dynamic scenes | PSNR: 26.02 -> 27.44 | Helped |
| Add flow consistency loss (Lflow) (ablation) | Base+BE+RW · NVIDIA dynamic scenes | PSNR: 27.44 -> 27.50 | Helped |
| Add depth loss (Ldepth) (ablation) | Ours+Lflow · NVIDIA dynamic scenes | PSNR: 27.50 -> 27.55 | Helped |
Limitations
- Relies on accurate camera poses and optical flow; inaccurate estimates cause motion artifacts and incorrect decoupling.
- Ghosting artifacts in motion areas and half-baked disentanglement results in failure cases.
- Per-scene optimization of multiple NeRFs with 7e5 iterations total; code only promised 'soon'.
Experimental setup
Datasets: NVIDIA dynamic scenes Argoverse
Benchmarks: NVIDIA dynamic scenes Urban driving scenes
Metrics: PSNR LPIPS SSIM
Baselines: Yoon et al. 2020 NeRF NSFF Gao et al. 2021 D2NeRF
Builds on: NeRF NSFF Gao et al. dynamic NeRF RAFT MiDaS depth estimation
Closest papers by meaning
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.89)
- NeRF-HuGS: Improved Neural Radiance Fields in Non-static Scenes Using Heuristics-Guided Segmentation (CVPR 2024 · similarity 0.88)
- BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling (AAAI 2024 · similarity 0.88)
- TeTriRF: Temporal Tri-Plane Radiance Fields for Efficient Free-Viewpoint Video (CVPR 2024 · similarity 0.88)
- DynaMoDe-NeRF: Motion-aware Deblurring Neural Radiance Field for Dynamic Scenes (CVPR 2025 · similarity 0.87)
- Multi-Level Neural Scene Graphs for Dynamic Urban Environments (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.