Research map / 3D vision / Gaussian splatting
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 4D Gaussian Splatting, a holistic representation that deforms a single canonical set of 3D Gaussians with a HexPlane-based spatial-temporal encoder and a tiny multi-head MLP decoder, enabling real-time dynamic scene rendering.
Problem
Real-time, memory- and training-efficient novel view synthesis of dynamic scenes, especially from monocular sparse inputs, where complex motion is hard to model efficiently.
Approach
Instead of per-frame 3D Gaussians or per-Gaussian motion tables, a single canonical set of 3D Gaussians is deformed at each timestamp by a Gaussian deformation field: a decomposed 4D neural voxel (HexPlane/K-Planes style) encodes features from nearby Gaussians and time, and a tiny multi-head MLP predicts position, rotation and scale offsets.
Key results
- On the synthetic dataset at 800x800, 4D-GS reaches 34.05 dB PSNR and 82 FPS with 18 MB storage and 8 min training.
- On Neu3D at 1352x1014, 4D-GS achieves 30 FPS with 31.15 PSNR and 40 min training, trading some PSNR versus Im4D (32.58) and MSTH (32.37).
- Ablation without the HexPlane encoder drops synthetic PSNR from 34.05 to 27.05, and removing the phi_x position head drops it to 26.67.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 4D-GS instead of V4D | Dynamic scene rendering · Synthesis dataset | PSNR (dB): 33.72 -> 34.05 | Helped |
| Use 4D-GS instead of V4D | Dynamic scene rendering · HyperNeRF vrig | PSNR (dB): 24.8 -> 25.2 | Helped |
| Use 4D-GS instead of HexPlane-all | Dynamic scene rendering · Neu3D | PSNR (dB): 31.70 -> 31.15 | Hurt |
| Use 4D-GS instead of MSTH | Dynamic scene rendering · Neu3D | FPS: 2(15‡) -> 30 | Helped |
| Add HexPlane encoder Rl(i, j) (ablation) | 4D-GS · Synthesis dataset | PSNR (dB): 27.05 -> 34.05 | Helped |
| Add 3D Gaussian warm-up initialization (ablation) | 4D-GS · Synthesis dataset | PSNR (dB): 31.91 -> 34.05 | Helped |
| Add position deformation head ϕx (ablation) | 4D-GS · Synthesis dataset | PSNR (dB): 26.67 -> 34.05 | Helped |
| Add rotation deformation head ϕr (ablation) | 4D-GS · Synthesis dataset | PSNR (dB): 33.08 -> 34.05 | Helped |
| Add scaling deformation head ϕs (ablation) | 4D-GS · Synthesis dataset | PSNR (dB): 33.02 -> 34.05 | Helped |
Surprising findings
- Removing the HexPlane encoder raises FPS to 140 but sharply lowers PSNR, showing a speed-quality trade-off.
- The vanilla 3D-GS baseline achieves only 23.19 dB on synthetic scenes because it lacks motion modeling.
Limitations
- Struggles with large motions, absence of background points, and inaccurate camera poses.
- Cannot separate static and dynamic Gaussians under monocular settings without extra supervision.
- Querying the deformation field for huge numbers of Gaussians hinders scaling to urban-scale reconstruction.
Experimental setup
Datasets: D-NeRF synthetic dataset HyperNeRF dataset Neu3D dataset
Benchmarks: D-NeRF synthetic HyperNeRF vrig Neu3D
Metrics: PSNR SSIM MS-SSIM LPIPS FPS training time storage
Baselines: TiNeuVox K-Planes HexPlane-Slim 3D-GS FFDNeRF MSTH V4D Im4D HyperReel NeRFPlayer HyperNeRF Nerfies
Builds on: 3D Gaussian Splatting HexPlane K-Planes D-NeRF
Closest papers by meaning
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.93)
- 1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering (NeurIPS 2025 · similarity 0.93)
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.92)
- Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (CVPR 2024 · similarity 0.92)
- Temporal Smoothness-Aware Rate-Distortion Optimized 4D Gaussian Splatting (NeurIPS 2025 · similarity 0.92)
- Grid4D: 4D Decomposed Hash Encoding for High-Fidelity Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.