Research map / 3D vision / Gaussian splatting
DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeSiRe-GS, a self-supervised 4D Gaussian splatting method that extracts 2D motion masks from rendering-vs-GT feature dissimilarity and distills them into PVG-based dynamic Gaussians for static-dynamic decomposition and surface reconstruction in driving scenes.
Problem
Self-supervised static-dynamic decomposition and high-fidelity surface reconstruction of urban driving scenes with 3D Gaussian splatting, without 3D bounding box annotations, under sparse views and sparse LiDAR depth.
Approach
3DGS reconstructs only static regions, so feature dissimilarity between rendered and GT images (from a frozen foundation model) yields 2D motion masks; a learned MLP decoder predicts dynamicness that is distilled into per-Gaussian velocities of PVG. Adds flattened disk-shaped Gaussians with normals derived from scale, giant-Gaussian penalty, and temporal cross-view depth consistency.
Key results
- On Waymo Open Dataset, reconstruction PSNR 33.61 vs PVG 32.46; on KITTI, reconstruction PSNR 33.94 vs PVG 32.83 (self-supervised SOTA in Table 1).
- Achieves about 36-41 FPS, slightly slower than PVG (50-59 FPS) and 3DGS, but over 1.1 PSNR gain on Waymo over PVG.
- Ablation shows removing stage I motion masks drops reconstruction PSNR from 35.76 to 34.71 (full model vs variant a).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DeSiRe-GS instead of PVG | Waymo Open Dataset image reconstruction · Waymo Open Dataset image reconstruction | PSNR: 32.46 -> 33.61 | Helped |
| add DeSiRe-GS instead of PVG | Waymo Open Dataset novel view synthesis · Waymo Open Dataset novel view synthesis | PSNR: 28.11 -> 29.75 | Helped |
| add DeSiRe-GS instead of PVG | KITTI image reconstruction · KITTI image reconstruction | PSNR: 32.83 -> 33.94 | Helped |
| add DeSiRe-GS instead of PVG | KITTI novel view synthesis · KITTI novel view synthesis | PSNR: 27.43 -> 28.87 | Helped |
| add stage I motion masks (ablation) | DeSiRe-GS · Waymo Open Dataset | PSNR: 34.7063 -> 35.7598 | Helped |
| use FiT3D instead of DINOv2 feature extractor (ablation) | DeSiRe-GS · Waymo Open Dataset | PSNR: 34.9551 -> 35.7598 | Helped |
| add ground-truth normal supervision (ablation) | DeSiRe-GS · Waymo Open Dataset | PSNR: 35.4469 -> 35.7598 | Helped |
| use ground-truth normal supervision instead of depth-derived normals (ablation) | DeSiRe-GS · Waymo Open Dataset | LPIPS: 0.1436 -> 0.0956 | Helped |
| add minimum-scale regularization (ablation) | DeSiRe-GS · Waymo Open Dataset | PSNR: 35.2863 -> 35.7598 | Helped |
| add maximum-scale regularization (ablation) | DeSiRe-GS · Waymo Open Dataset | PSNR: 35.6911 -> 35.7598 | No clear effect |
| add multi-view consistency (ablation) | DeSiRe-GS · Waymo Open Dataset | Depth L1: 0.1154 -> 0.0713 | Helped |
Surprising findings
- Pseudo-normals from depth gradients slightly improve depth accuracy but reduce rendering quality compared with Omnidata normals.
- Scale regularization gives only marginal rendering gains, attributed to Gaussians' strong overfitting capacity, though it improves 3D structure for decomposition.
Limitations
- Rendering speed slower than 3DGS and PVG baselines due to extra attributes such as normals
- Scale regularization gains on rendering metrics are not significant
Experimental setup
Datasets: Waymo Open Dataset KITTI
Benchmarks: Waymo Open Dataset KITTI
Metrics: PSNR SSIM LPIPS DPSNR DSSIM Depth L1 FPS
Baselines: 3DGS StreetSurf Mars SUDS EmerNeRF S3Gaussian PVG OmniRe StreetGS HUGS
Builds on: PVG 3D Gaussian Splatting 2D Gaussian Splatting FiT3D
Closest papers by meaning
- DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes (CVPR 2024 · similarity 0.91)
- DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Input (AAAI 2025 · similarity 0.90)
- SplatFlow: Self-Supervised Dynamic Gaussian Splatting in Neural Motion Flow Field for Autonomous Driving (CVPR 2025 · similarity 0.90)
- Horizon-GS: Unified 3D Gaussian Splatting for Large-Scale Aerial-to-Ground Scenes (CVPR 2025 · similarity 0.89)
- EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesis (CVPR 2025 · similarity 0.88)
- SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.