Research map / 3D vision / Gaussian splatting
Superpoint Gaussian Splatting for Real-Time High-Fidelity Dynamic Scene Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Superpoint Gaussian Splatting (SP-GS), which clusters 3D Gaussians into superpoints that share a rigid deformation predicted by a tiny MLP, enabling real-time dynamic scene rendering with high fidelity.
Problem
Novel view synthesis for dynamic scenes is slow and lower quality with NeRF-based methods using time-variant MLPs; per-Gaussian deformation networks in 3D-GS-based extensions greatly reduce rendering speed.
Approach
Learn a soft association matrix that groups canonical 3D Gaussians into a small number of superpoints, predict rigid rotation and translation per superpoint with a deformation MLP, and propagate them to Gaussians, with a property reconstruction loss enforcing as-rigid-as-possible grouping.
Key results
- On D-NeRF, SP-GS reaches 37.98 PSNR at 227.25 FPS at 800x800, versus 40.23 PSNR and 45.05 FPS for D-3D-GS.
- On HyperNeRF (536x960), SP-GS reaches 117.86 FPS with 25.61 PSNR, compared with 66.21 FPS for 4D-GS (25.02 PSNR).
- On NeRF-DS, SP-GS achieves 251.70 FPS versus 15.27 FPS for D-3D-GS, with PSNR 23.15 vs 23.78.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the non-rigid deformation network | SP-GS · D-NeRF | FPS: 227.25 -> 119.35 | Hurt |
| Add the non-rigid deformation network | SP-GS · HyperNeRF | PSNR: 25.61 -> 26.78 | Helped |
| Add the non-rigid deformation network | SP-GS · NeRF-DS | PSNR: 23.15 -> 23.33 | No clear effect |
| Add the non-rigid deformation network | SP-GS · NeRF-DS | FPS: 251.70 -> 66.13 | Hurt |
| Add property reconstruction loss (ablation) | SP-GS · D-NeRF | PSNR: 37.59 -> 37.98 | Helped |
| Add property reconstruction loss (ablation) | SP-GS · D-NeRF | LPIPS: 0.0172 -> 0.0164 | Helped |
| Use a warm-up training stage (ablation) | SP-GS · D-NeRF | PSNR: 21.56 -> 37.98 | Helped |
| Use interpolation instead of the deformation network at inference (ablation) | SP-GS · D-NeRF | FPS: 168.01 -> 219.95 | Helped |
| Use interpolation instead of the deformation network at inference (ablation) | SP-GS · D-NeRF | PSNR: 36.2281 -> 36.2280 | No clear effect |
| Increase deformation network width from 64 to 256 at depth 8 (ablation) | SP-GS · D-NeRF | PSNR: 35.8021 -> 36.4452 | Helped |
| Vary the number of superpoints from 50 to 500 (ablation) | SP-GS · D-NeRF | PSNR: 35.69 to 36.52 | No clear effect |
Surprising findings
- SP-GS trails the D-3D-GS heavy per-Gaussian deformation baseline in PSNR on D-NeRF and NeRF-DS, trading some quality for speed.
- Removing the warm-up stage collapses PSNR from 37.98 to 21.56 on D-NeRF.
Limitations
- Relies on COLMAP-based sparse point cloud initialization, which fails for dynamic scenes and degrades camera poses and convergence.
- Rigid-motion assumption per superpoint; non-rigid motion needs an optional extra network (SP-GS+NG) that lowers FPS.
- Under-performs D-3D-GS on some metrics (PSNR on D-NeRF and NeRF-DS).
Experimental setup
Datasets: D-NeRF HyperNeRF NeRF-DS NVIDIA Dynamic Scene Dataset
Benchmarks: D-NeRF HyperNeRF NeRF-DS NVIDIA Dynamic Scene Dataset
Metrics: PSNR SSIM MS-SSIM LPIPS FPS
Baselines: D-NeRF TiNeuVox K-Planes HexPlane Tensor4D TI-DNeRF NeRFPlayer 4D-GS D-3D-GS 3D-GS
Builds on: 3D Gaussian Splatting Deformable 3D Gaussians HexPlane NeRF
Closest papers by meaning
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
- Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation (AAAI 2025 · similarity 0.92)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.92)
- Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (CVPR 2024 · similarity 0.91)
- SplatFormer: Point Transformer for Robust 3D Gaussian Splatting (ICLR 2025 · similarity 0.91)
- IBGS: Image-Based Gaussian Splatting (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.