Research map / 3D vision / Gaussian splatting
OmniRe: Omni Urban Scene Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds an OmniRe Gaussian scene graph on 3DGS with rigid, SMPL-driven, and shared-deformation nodes to reconstruct and simulate all dynamic urban actors from driving logs.
Problem
Existing Gaussian and neural-field methods for urban scene reconstruction mainly model rigid vehicles, neglecting non-rigid road users such as pedestrians and cyclists, which limits holistic and interactive simulation.
Approach
A Gaussian scene graph in which vehicles are rigid Gaussian nodes, pedestrians are SMPL-driven Gaussian nodes with jointly refined poses estimated from multi-camera 4D-Humans outputs, and other non-rigid actors use a shared per-node deformation network disambiguated by instance embeddings, with boxes and poses optimized jointly.
Key results
- On the 8-scene Waymo comparison, OmniRe reaches 34.25 full-image PSNR for reconstruction and 32.57 for novel view synthesis, versus 32.37 and 30.19 for the best baseline PVG.
- Human-region reconstruction PSNR is 28.15 versus 24.06 for PVG, a gain of about 4.09 dB.
- On 32 Waymo scenes, OmniRe reaches 33.73 full-image PSNR against 31.29 for EmerNeRF and 29.93 for StreetGS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SMPL actor modeling (ablation) | OmniRe · Waymo scene reconstruction, human | PSNR: 24.71 -> 28.15 | Helped |
| add SMPL actor modeling (ablation) | OmniRe · Waymo NVS, human | PSNR: 23.18 -> 24.36 | Helped |
| refine human body poses jointly with Gaussians (ablation) | OmniRe · Waymo scene reconstruction, human | PSNR: 26.97 -> 28.15 | Helped |
| add deformable actor nodes (ablation) | OmniRe · Waymo NVS, human | PSNR: 22.41 -> 24.36 | Helped |
| refine instance bounding boxes during training (ablation) | OmniRe · Waymo scene reconstruction, vehicle | PSNR: 25.57 -> 28.91 | Helped |
| enable AbsGrad densification (ablation) | OmniRe · Waymo scene reconstruction, full image | PSNR: 34.11 -> 34.25 | Helped |
Surprising findings
- Disabling AbsGrad densification costs only about 0.1 PSNR across methods, so it is not the source of OmniRe's advantage.
Limitations
- Does not explicitly model lighting effects, which can cause visual inconsistency when combining assets captured under different lighting.
- Per-scene optimization yields poorer novel views when the camera deviates far from training trajectories.
Experimental setup
Datasets: Waymo Open Dataset NuScenes Argoverse2 PandaSet KITTI NuPlan
Benchmarks: Waymo Open Dataset
Metrics: PSNR SSIM RMSE Chamfer Distance
Baselines: 3DGS DeformableGS StreetGS HUGS PVG EmerNeRF
Builds on: 3D Gaussian Splatting SMPL PVG 4D-Humans neural scene graphs
Closest papers by meaning
- Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstruction (CVPR 2025 · similarity 0.86)
- FRUC: Feedforward Dynamic Scene Reconstruction from Uncalibrated Collaborative Driving Views (NeurIPS 2026 · similarity 0.85)
- DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes (ICLR 2025 · similarity 0.85)
- Multi-Level Neural Scene Graphs for Dynamic Urban Environments (CVPR 2024 · similarity 0.85)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.85)
- Dynamic 3D Gaussian Fields for Urban Areas (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.