Research map / 3D vision / Gaussian splatting
Dynamic 3D Gaussian Fields for Urban Areas
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 4DGF, a hybrid scene representation combining 3D Gaussians as a geometry scaffold, hash-grid neural fields for appearance and transient geometry, and a scene graph with deformations for dynamic objects, enabling large-scale heterogeneous urban NVS at interactive rates.
Problem
Novel-view synthesis of large-scale, dynamic urban areas from heterogeneous captures (varying weather, season, lighting) at interactive rendering speeds, which prior NeRF and 3DGS methods cannot handle.
Approach
Use 3D Gaussians without per-primitive appearance as a geometry scaffold, with hash-grid neural fields predicting per-sequence color and opacity attenuation (transient geometry), a scene graph of sequence/object nodes for global rigid motion, and canonical-space deformations for non-rigid objects, plus multi-GPU ADC and pose optimization.
Key results
- On Argoverse 2 multi-sequence NVS, 4DGF reaches 24.97 mean PSNR versus 21.15 for ML-NSG (over 3 dB better) with 0.074 s render time versus 21.7 s.
- On KITTI [75%], 4DGF achieves 31.34 PSNR and 0.945 SSIM, outperforming StreetGaussians (25.79 PSNR).
- On Waymo Dynamic-32, 4DGF reaches 29.04 full-image PSNR and 29.34 dynamic-only PSNR, with 0.025 s rendering, about 700x faster than the best alternative.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 4DGF instead of ML-NSG | Argoverse 2 Residential + Downtown (Mean) · Argoverse 2 | PSNR: 21.15 -> 24.97 | Helped |
| Use 4DGF instead of ML-NSG | Argoverse 2 Residential + Downtown (Mean) · Argoverse 2 | render time: 21.7 -> 0.074 | Helped |
| Use dynamic modeling (ablation) | 4DGF, single-sequence · Residential split | PSNR: 24.31 -> 28.55 | Helped |
| Use neural fields instead of per-Gaussian appearance attributes (ablation) | 4DGF, single-sequence · Residential split | GPU memory: 8.6 GB -> 4.5 GB | Helped |
| Add the generated 3D Gaussian background (ablation) | 4DGF, single-sequence · Residential split | PSNR: 28.49 -> 28.81 | Helped |
| Add sequence appearance modeling (ablation) | 4DGF, multi-sequence · Residential split | PSNR: 22.37 -> 25.13 | Helped |
| Add transient-geometry modeling (ablation) | 4DGF, multi-sequence · Residential split | PSNR: 25.13 -> 25.78 | Helped |
| Add deformation head χ for non-rigid motion (ablation) | 4DGF · Waymo Open Dynamic-32 subset (12 sequences with most non-rigid objects), Full Image | PSNR: 29.52 -> 29.60 | Helped |
| Use modified ADC instead of vanilla ADC (ablation) | 4DGF · Residential split, single-sequence | PSNR: 28.60 -> 28.81 | Helped |
| Use predicted rather than ground-truth 3D boxes (ablation) | 4DGF · Residential split, single-sequence | PSNR: 28.81 -> 28.52 | Hurt |
| Use predicted rather than ground-truth 3D boxes (ablation) | 4DGF · Residential split, multi-sequence | PSNR: 25.78 -> 25.67 | Hurt |
| Add regularization on 3D Gaussian scales (ablation) | 4DGF · Urban scene reconstruction | view synthesis metrics | No clear effect |
Surprising findings
- Modeling per-Gaussian appearance with neural fields is on par with spherical harmonics in quality while halving peak training GPU memory (8.6 GB to 4.5 GB).
- Neural field queries account for only 13.3% of average runtime, with scene graph evaluation and rasterization dominating.
Limitations
- Assumes a pinhole camera model and does not handle distortions such as rolling shutter, white balance, or blur.
- Does not model other sensors such as LiDAR or non-pinhole cameras like equirectangular.
- Large Gaussians can cause fog-like artifacts in free-viewpoint renderings.
Experimental setup
Datasets: Argoverse 2 KITTI VKITTI2 Waymo Open
Benchmarks: Argoverse 2 NVS benchmark KITTI VKITTI2 Waymo Open Dynamic-32
Metrics: PSNR SSIM LPIPS render time
Baselines: SUDS ML-NSG Nerfacto NSG MARS StreetGaussians EmerNeRF NeuRAD
Builds on: 3D Gaussian splatting Neural Scene Graphs Instant-NGP EmerNeRF SUDS
Closest papers by meaning
- Multi-Level Neural Scene Graphs for Dynamic Urban Environments (CVPR 2024 · similarity 0.91)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.88)
- DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes (ICLR 2025 · similarity 0.88)
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (CVPR 2024 · similarity 0.88)
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.87)
- Swift-Mapping: Online Neural Implicit Dense Mapping in Urban Scenes (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.