Research map / 3D vision / 3D generation
EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EmerNeRF, a self-supervised hybrid static/dynamic neural field with emergent scene flow and PE-free lifting of ViT foundation features for 4D driving scene reconstruction, evaluated on the new NOTR benchmark.
Problem
Learning 4D spatial-temporal representations of dynamic driving scenes from a single traversal with sparse, transient observations, without ground-truth object annotations or pre-trained segmentation/flow models.
Approach
Decompose the scene into static and dynamic hash-grid fields regularized by dynamic density, add a self-supervised scene flow field whose flows drive multi-frame feature aggregation so flow emerges from reconstruction losses, and lift DINO features to 4D while removing ViT positional-embedding artifacts with a learnable shared PE map.
Key results
- On NOTR Dynamic-32 scene reconstruction, EmerNeRF reaches 28.87 PSNR full image vs 25.17 for HyperNeRF, and 26.19 dynamic-only PSNR vs 22.93.
- Scene flow on Dynamic-32: EPE3D 0.014 m and Acc5 93.92% vs 0.365 m and 51.76% for NSFP.
- Removing DINOv2 positional-embedding patterns gives an average 37.50% relative micro-accuracy improvement in few-shot semantic occupancy prediction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EmerNeRF instead of HyperNeRF | Dynamic-32 scene reconstruction · Dynamic-32 scene reconstruction, full image | PSNR: 25.17 -> 28.87 | Helped |
| Use EmerNeRF instead of HyperNeRF | Dynamic-32 scene reconstruction · Dynamic-32 scene reconstruction, dynamic-only | PSNR: 22.93 -> 26.19 | Helped |
| Use EmerNeRF instead of StreetSurf | Static-32 scene reconstruction · Static-32 scene reconstruction | PSNR: 26.15 -> 29.08 | Helped |
| Use EmerNeRF instead of NSFP | Dynamic-32 scene flow estimation · Dynamic-32 | EPE3D (m): 0.365 -> 0.014 | Helped |
| Remove positional embedding patterns | EmerNeRF with DINOv2 features · Average of 3 splits semantic occupancy | Micro Acc: +37.50% relative improvement | Helped |
| Remove positional embedding patterns | EmerNeRF with DINOv2 features · Static-32 feature synthesis | Feature-PSNR: 21.87 -> 22.70 (+0.83) | Helped |
| Add a static field rather than use full 4D iNGP without one (ablation) | EmerNeRF · Dynamic-32 scene reconstruction, full image | PSNR: 26.55 -> 27.21 | Helped |
| Add a flow field (ablation) | EmerNeRF · Dynamic-32 scene reconstruction, full image | PSNR: 26.92 -> 27.21 | Helped |
| Add temporal feature aggregation (ablation) | EmerNeRF · Dynamic-32 scene flow | Acc5 (%): 4.53% -> 89.74% | Helped |
| Allow gradients through temporally displaced features during aggregation (ablation) | EmerNeRF · Dynamic-32 scene flow | Acc5 (%): 3.87% -> 89.74% | Helped |
Surprising findings
- Scene flow emerges from reconstruction losses alone; disabling temporal aggregation or freezing temporal feature gradients removes flow ability.
- Even DINOv1 features, which look visually unaffected, showed a benefit from PE removal in 4D occupancy prediction.
Limitations
- Does not optimize camera poses and is prone to rolling shutter effects of camera and LiDAR.
- Struggles to estimate motion of slow-moving objects when the ego vehicle moves fast.
- Balance between geometry and rendering quality remains a trade-off.
Experimental setup
Datasets: Waymo Open Dataset (scene flow version) Occ3D
Benchmarks: NOTR Dynamic-32 NOTR Static-32 NOTR Diverse-56 Occ3D
Metrics: PSNR SSIM dynamic-only PSNR EPE3D Acc5 Acc10 angle error micro accuracy macro accuracy feature PSNR
Baselines: D^2NeRF HyperNeRF StreetSurf iNGP NSFP
Builds on: iNGP StreetSurf HyperNeRF D^2NeRF nerfacc
Closest papers by meaning
- Flux4D: Flow-based Unsupervised 4D Reconstruction (NeurIPS 2025 · similarity 0.87)
- Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling (CVPR 2024 · similarity 0.87)
- DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features (NeurIPS 2024 · similarity 0.86)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.86)
- SplatFlow: Self-Supervised Dynamic Gaussian Splatting in Neural Motion Flow Field for Autonomous Driving (CVPR 2025 · similarity 0.86)
- DaReNeRF: Direction-aware Representation for Dynamic Scenes (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.