Research map / 3D vision / Gaussian splatting
Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Spacetime Gaussians (3D Gaussians extended with temporal opacity and polynomial motion/rotation) with splatted neural feature rendering and error/depth-guided Gaussian sampling for real-time, compact multi-view dynamic view synthesis.
Problem
Novel view synthesis of dynamic real-world scenes that jointly achieves photorealistic quality, real-time high-resolution rendering, and compact storage.
Approach
Extend 3D Gaussians to 4D with a temporal radial basis opacity and polynomial position/rotation trajectories, replace spherical harmonics with compact per-Gaussian features decoded by a tiny MLP, and add guided sampling of new Gaussians where training error is high and coarse depth indicates range.
Key results
- On the Neural 3D Video Dataset, the method reaches 32.05 PSNR at 140 FPS with 200 MB total model size for 300 frames, with best LPIPS (0.044) among compared methods.
- On the Google Immersive Dataset, it reports 29.2 PSNR and 99 FPS versus HyperReel's 28.8 PSNR and 4 FPS, at 1.2 MB per frame.
- On the Technicolor Dataset, it reports 33.6 PSNR at 86.7 FPS and 1.1 MB per frame, outperforming HyperReel (32.7 PSNR, 4 FPS).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Spacetime Gaussian Feature Splatting instead of HexPlane | Dynamic view synthesis · Neural 3D Video Dataset | PSNR: 31.71 -> 32.74 | Helped |
| use Spacetime Gaussian Feature Splatting instead of HyperReel | Dynamic view synthesis · Google Immersive Dataset | PSNR: 28.8 -> 29.2 | Helped |
| use Spacetime Gaussian Feature Splatting instead of HyperReel | Dynamic view synthesis · Technicolor Dataset | PSNR: 32.7 -> 33.6 | Helped |
| add temporal opacity (ablation) | Spacetime Gaussian Feature Splatting · Technicolor Dataset | PSNR: 31.0 -> 33.6 | Helped |
| add polynomial motion (ablation) | Spacetime Gaussian Feature Splatting · Technicolor Dataset | PSNR: 32.6 -> 33.6 | Helped |
| add polynomial rotation (ablation) | Spacetime Gaussian Feature Splatting · Technicolor Dataset | PSNR: 33.4 -> 33.6 | Helped |
| add feature splatting for view- and time-dependent appearance (ablation) | Spacetime Gaussian Feature Splatting · Technicolor Dataset | PSNR: 33.0 -> 33.6 | Helped |
| add guided sampling of Gaussians (ablation) | Spacetime Gaussian Feature Splatting · Technicolor Dataset | PSNR: 33.3 -> 33.6 | Helped |
| initialize with SfM point clouds every 4 frames instead of every 1 frame (ablation) | Spacetime Gaussian Feature Splatting · Technicolor Dataset, Theater | PSNR: 31.58 -> 31.51 | Hurt |
| initialize with SfM point clouds every 16 frames instead of every 1 frame (ablation) | Spacetime Gaussian Feature Splatting · Technicolor Dataset, Theater | PSNR: 31.58 -> 31.04 | Hurt |
Surprising findings
- Removing guided sampling caused little performance drop on Technicolor, since scenes are richly textured and well covered by SfM points.
- Using SfM points from fewer frames for initialization slightly lowers quality but significantly reduces model size.
Limitations
- Cannot be trained on-the-fly, limiting streaming applications.
- Currently restricted to multi-view video inputs; monocular setting not addressed.
Experimental setup
Datasets: Neural 3D Video Dataset Google Immersive Dataset Technicolor Dataset
Benchmarks: Neural 3D Video Dataset Google Immersive Dataset Technicolor Dataset
Metrics: PSNR DSSIM LPIPS FPS model size
Baselines: DyNeRF HyperReel NeRFPlayer K-Planes MixVoxels-L MixVoxels-X HexPlane StreamRF Neural Volumes LLFF
Builds on: 3D Gaussian Splatting NeuRBF HyperReel K-Planes
Closest papers by meaning
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.92)
- Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modeling (AAAI 2025 · similarity 0.92)
- 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering (CVPR 2024 · similarity 0.92)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.91)
- Superpoint Gaussian Splatting for Real-Time High-Fidelity Dynamic Scene Reconstruction (ICML 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.