Research map / 3D vision / Gaussian splatting
Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Instant Gaussian Stream (IGS), a generalizable anchor-driven network that predicts Gaussian motion between frames in one forward pass, plus key-frame-guided refinement, enabling ~2.7 s per-frame streaming reconstruction of dynamic scenes.
Problem
Streaming free-viewpoint video reconstruction with 3D Gaussian Splatting requires per-frame optimization (10s+ per frame) and suffers from error accumulation across frames.
Approach
A generalized Anchor-driven Gaussian Motion Network lifts multi-view 2D optical-flow motion features onto FPS-sampled anchor points via projection, transforms them with a Transformer, and decodes per-Gaussian position/rotation deltas in a single feedforward pass; periodic key frames are refined with a max-points-bounded optimization to stop error accumulation.
Key results
- IGS-s reaches 33.89 dB PSNR on N3DV with 2.67 s per-frame training and 204 FPS, versus 33.11 dB for 3DGStream at 12 s.
- IGS-l reaches 34.15 dB PSNR on N3DV at 3.35 s per frame.
- Cross-domain on Meeting Room (trained on N3DV): IGS-s 29.24 dB at 2.77 s/frame vs 3DGStream 28.36 dB at 11.51 s/frame.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IGS-s instead of 3DGStream† | N3DV streaming reconstruction · N3DV | PSNR: 32.75 -> 33.89 | Helped |
| Use IGS-s instead of 3DGStream† | N3DV streaming reconstruction · N3DV | Train time: 16.93 -> 2.67 | Helped |
| Use IGS-s instead of 3DGStream† | Meeting Room streaming reconstruction · Meeting Room | PSNR: 28.36 -> 29.24 | Helped |
| Use IGS-s instead of 3DGStream† | Meeting Room streaming reconstruction · Meeting Room | Train time: 11.51 -> 2.77 | Helped |
| Replace the pretrained optical-flow model with a jointly trained 4-layer UNet (ablation) | IGS · N3DV | PSNR: 33.62 -> 31.07 | Hurt |
| Replace projection-aware feature lift with Transformer cross-attention (ablation) | IGS · N3DV | PSNR: 33.62 -> 32.95 | Hurt |
| Remove the maximum-point bound during refinement (ablation) | IGS · N3DV | Storage: 7.90 -> 110.26 | Hurt |
| Remove the maximum-point bound during refinement (ablation) | IGS · N3DV | PSNR: 33.62 -> 33.23 | Hurt |
| Set the key-frame interval to w=1 (ablation) | IGS · N3DV | Train time: 2.67 -> 6.38 | Hurt |
| Set the key-frame interval to w=1 (ablation) | IGS · N3DV | Storage: 7.90 -> 36.0 | Hurt |
| Set the key-frame interval to w=10 (ablation) | IGS · N3DV | PSNR: 33.62 -> 30.14 | Hurt |
| Use 100 rather than 50 key-frame refinement iterations | IGS · N3DV | PSNR: 33.89 -> 34.15 | Helped |
Surprising findings
- Max-points bounding of key-frame refinement slightly lowers PSNR cost trade-off is not monotonic: w=1 keyframes overfit and reduce test quality.
- IGS shows more per-frame PSNR fluctuation than 3DGStream, since 3DGStream's small-motion assumption yields smoother output.
Limitations
- Jitter between adjacent frames due to lack of temporal dependencies in the framework
- Fails to capture large motion relative to 3DGStream's smoothness; sensitive to noise
- Key-frame refinement still requires per-keyframe optimization
Experimental setup
Datasets: N3DV Meeting Room
Benchmarks: N3DV Meeting Room
Metrics: PSNR train time per frame render FPS storage
Baselines: 3DGStream StreamRF Kplanes 4DGS Spacetime-GS SaRo-GS Realtime-4DGS
Builds on: 3D Gaussian Splatting 3DGStream GM-Flow LightGaussian Rade-GS
Closest papers by meaning
- Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction (NeurIPS 2025 · similarity 0.94)
- HiCoM: Hierarchical Coherent Motion for Dynamic Streamable Scenes with 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
- 3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos (CVPR 2024 · similarity 0.91)
- Compressing Streamable Free-Viewpoint Videos to 0.1 MB per Frame (AAAI 2025 · similarity 0.90)
- Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle (CVPR 2024 · similarity 0.90)
- Superpoint Gaussian Splatting for Real-Time High-Fidelity Dynamic Scene Reconstruction (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.