Research map / 3D vision / Gaussian splatting
Instant4D: 4D Gaussian Splatting in Minutes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Instant4D reconstructs dynamic scenes from uncalibrated monocular video in minutes by combining MegaSAM visual SLAM, grid (voxel) pruning of back-projected depth, and a simplified isotropic motion-aware 4D Gaussian representation.
Problem
Dynamic novel view synthesis from casual, uncalibrated monocular video is slow, often requiring hours to days of optimization and complex pose/parameter estimation.
Approach
Decouple geometry recovery (MegaSAM visual SLAM plus video-consistent depth, back-projected to a point cloud) from photometric optimization, prune the point cloud with adaptive voxel grid filtering, and model Gaussians as isotropic 4D primitives with RGB color and motion-aware temporal scales instead of high-order SCH.
Key results
- On DyCheck iPhone, the Full model reaches 24.52 dB average PSNR, 7.15 dB above RoDyGS and 1.89 dB above Deform3D, with about 7.2 minutes training.
- On the NVIDIA dataset, Ours reaches 23.99 PSNR with 0.02 h runtime and 822 FPS at 480x270, versus 4DGS at 21.45 PSNR and 1.2 h.
- Grid pruning cuts the NVIDIA model memory from 10.7 GB to 0.83 GB (92%) and training time from 181 s to 42 s (about 4x).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Instant4D instead of InstantSplat | NVIDIA Dynamic Scene reconstruction · NVIDIA Dynamic Scene | Rendering FPS at 480 × 270: 117 -> 822 | Helped |
| Use Instant4D (Full) instead of Deform3D | DyCheck iPhone reconstruction · DyCheck iPhone | Average PSNR: 22.63 -> 24.52 | Helped |
| Add simple RGB appearance instead of SCH (ablation) | Instant4D on NVIDIA Dynamic Scene · NVIDIA Dynamic Scene | PSNR: 23.83 -> 23.99 | Helped |
| Add grid pruning (ablation) | Instant4D on NVIDIA Dynamic Scene · NVIDIA Dynamic Scene | Runtime (sec): 181 -> 42 | Helped |
| Add grid pruning (ablation) | Instant4D on NVIDIA Dynamic Scene · NVIDIA Dynamic Scene | PSNR: 23.38 -> 23.99 | Helped |
| Add motion-aware temporal scaling (ablation) | Instant4D (Full) · DyCheck iPhone, 2× resolution | PSNR: 21.11 -> 24.52 | Helped |
| Use isotropic rather than anisotropic Gaussian covariance (ablation) | Instant4D (Full) · DyCheck iPhone, 2× resolution | PSNR: 23.00 -> 24.52 | Helped |
| Use zero-degree SH (ablation) | Instant4D (Full) · DyCheck iPhone, 2× resolution | PSNR: 23.52 -> 24.52 | Helped |
Surprising findings
- Replacing the simplified RGB color with 4D spherical-cylindrical harmonics slightly lowers PSNR (by 1.0 dB on DyCheck) rather than improving it.
- Off-the-shelf DUSt3R-family models (MonST3R, CUT3R) were rejected because their point clouds shift frequently over long sequences, so MegaSAM was used instead.
Limitations
- Memory grows linearly with sequence length because per-frame depth maps are retained, limiting long videos.
- Depth estimation is unstable for highly reflective or transparent surfaces.
- Low-texture scenes (e.g., Kite-surf) cause inaccurate SLAM poses and missing subjects in renders.
Experimental setup
Datasets: NVIDIA Dynamic Scene dataset DyCheck iPhone dataset DAVIS
Benchmarks: NVIDIA Dynamic Scene DyCheck iPhone
Metrics: PSNR SSIM runtime rendering FPS peak memory
Baselines: HyperNeRF DynamicNeRF RoDynRF 4DGS Casual-FVS InstantSplat D-NeRF RoDyGS Deform3D
Builds on: 4D Gaussian Splatting MegaSAM DROID-SLAM 3D Gaussian Splatting Gaussian Marbles
Closest papers by meaning
- 4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization (NeurIPS 2024 · similarity 0.92)
- 1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering (NeurIPS 2025 · similarity 0.90)
- Swift4D: Adaptive divide-and-conquer Gaussian Splatting for compact and efficient reconstruction of dynamic scene (ICLR 2025 · similarity 0.90)
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.90)
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.90)
- Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.