Research map / 3D vision / Gaussian splatting
4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UA-4DGS, a 4D Gaussian Splatting method for casually captured monocular videos that uses uncertainty-weighted diffusion and depth-smoothness regularization plus dynamic-region densification, and also extends to few-shot static scenes (UA-3DGS).
Problem
4D Gaussian Splatting overfits training frames when reconstructing dynamic scenes from casually recorded, fast-motion monocular handheld videos, and SfM initialization leaves fast-moving dynamic regions without Gaussian primitives.
Approach
Estimate per-Gaussian uncertainty from accumulated blending contributions over training views, render a 2D uncertainty map for novel views, and apply diffusion-prior and depth-TV regularization weighted by this map so that well-observed regions are not over-smoothed; initialize extra Gaussians in dynamic regions from estimated depth and RGB.
Key results
- On DyCheck, UA-4DGS reaches 15.25 mPSNR, 0.488 mSSIM, 0.390 mLPIPS versus 14.14 / 0.465 / 0.430 for the 4DGS baseline.
- On the spin scene, the ablation shows mPSNR rising from 15.25 (4DGS) to 17.37 with the full method.
- On LLFF with three training views, UA-3DGS reaches 20.89 PSNR and 0.217 LPIPS, versus 19.82 and 0.238 for FSGS reproduced.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add data-driven depth and flow losses (ablation) | 4DGS · DyCheck spin | mPSNR: 15.25 -> 15.74 | Helped |
| add dynamic region densification (ablation) | Ours + data-driven losses · DyCheck spin | mPSNR: 15.74 -> 17.04 | Helped |
| add uncertainty-aware diffusion loss (ablation) | Ours + data-driven losses + dynamic region densification · DyCheck spin | mPSNR: 17.04 -> 17.32 | Helped |
| add uncertainty-aware depth total variation loss (ablation) | Ours + data-driven losses + dynamic region densification + UA-diff · DyCheck spin | mPSNR: 17.32 -> 17.37 | Helped |
| use uncertainty-aware diffusion regularization instead of uniform diffusion regularization (ablation) | FSGS · LLFF room (train) | PSNR: 38.40 -> 41.28 | Helped |
| use uncertainty-aware diffusion regularization instead of uniform diffusion regularization (ablation) | FSGS · LLFF room (test) | PSNR: 20.40 -> 20.98 | Helped |
| use uncertainty-aware TV regularization instead of uniform TV regularization (ablation) | FSGS · LLFF room (train) | PSNR: 38.25 -> 38.26 | No clear effect |
| use uncertainty-aware TV regularization instead of uniform TV regularization (ablation) | FSGS · LLFF room (test) | PSNR: 20.77 -> 21.08 | Helped |
| use UA-4DGS instead of 4DGS | 4D Gaussian Splatting · DyCheck | mPSNR: 14.14 -> 15.25 | Helped |
| use UA-3DGS instead of FSGS | FSGS · LLFF (3 views) | PSNR: 19.82 -> 20.89 | Helped |
Surprising findings
- Gaussian-splatting methods fall behind MLP-based methods on the casual monocular DyCheck setting.
- Uniform diffusion and TV regularization degrade training-image reconstruction, while uncertainty-weighted versions preserve it.
Limitations
- Performance on casually recorded monocular videos still lags behind multi-view or simpler datasets.
- Regularization operates at the image level with 2D uncertainty maps rather than per-Gaussian.
- Dynamic region densification does not enforce temporal consistency in primitive initialization.
Experimental setup
Datasets: DyCheck LLFF
Benchmarks: DyCheck LLFF
Metrics: mPSNR mSSIM mLPIPS PSNR SSIM LPIPS FPS
Baselines: D-3DGS Zhan et al. 4DGS T-NeRF NSFF Nerfies HyperNeRF FSGS 3DGS DietNeRF RegNeRF FreeNeRF
Builds on: 4DGS (Wu et al. 4D Gaussian splatting with Hexplane) FSGS 3D Gaussian Splatting SDEdit DreamBooth
Closest papers by meaning
- Instant4D: 4D Gaussian Splatting in Minutes (NeurIPS 2025 · similarity 0.92)
- MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors (ICLR 2025 · similarity 0.92)
- KeyGS: A Keyframe-Centric Gaussian Splatting Method for Monocular Image Sequences (AAAI 2025 · similarity 0.91)
- Swift4D: Adaptive divide-and-conquer Gaussian Splatting for compact and efficient reconstruction of dynamic scene (ICLR 2025 · similarity 0.91)
- Orientation-anchored Hyper-Gaussian for 4D Reconstruction from Casual Videos (NeurIPS 2025 · similarity 0.91)
- Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.