Research map / 3D vision / Gaussian splatting
Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ComGS, a keypoint-driven motion representation for storage-efficient online free-viewpoint video reconstruction with 3D Gaussian Splatting, using motion-sensitive keypoint selection, an adaptive influence field, and error-aware key frame correction.
Problem
Online free-viewpoint video reconstruction with 3DGS requires prohibitive storage and bandwidth because existing methods model per-Gaussian (point-wise) attribute residuals rather than exploiting the locality and consistency of motion.
Approach
Select a sparse set of motion-sensitive keypoints via viewspace gradient differences between frames, let each keypoint drive neighboring Gaussians through a learned spatial influence field (instead of KNN), and correct only error-prone Gaussians at key frames using a learned soft/hard mask.
Key results
- ComGS-s achieves 0.049 MB storage on N3DV versus 7.80 MB for 3DGStream (about 159x smaller) and 0.68 MB for QUEEN-s, with PSNR 31.87 dB vs 31.67 dB for 3DGStream.
- On MeetRoom, ComGS-s reaches 31.49 dB PSNR and 0.028 MB storage, versus 30.79 dB and 4.1 MB for 3DGStream.
- Ablation shows adaptive influence-field control beats KNN control (31.87 vs 31.39 dB PSNR) and error-aware correction reduces storage from 373 KB to 49 KB with higher PSNR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ComGS-s instead of 3DGStream | Online FVV reconstruction · N3DV | Storage (MB): 7.80 -> 0.049 | Helped |
| Use ComGS-s instead of QUEEN-s | Online FVV reconstruction · MeetRoom | PSNR (dB): 31.14 -> 31.49 | Helped |
| Use ComGS-s instead of QUEEN-s | Online FVV reconstruction · MeetRoom | Storage (MB): 0.45 -> 0.028 | Helped |
| Add motion-sensitive keypoint selection instead of random selection (ablation) | ComGS · N3DV flame steak | PSNR (dB): 33.27 -> 33.49 | Helped |
| Add motion-sensitive keypoint selection instead of random selection (ablation) | ComGS · N3DV flame salmon | PSNR (dB): 29.22 -> 29.32 | Helped |
| Add adaptive motion-driven mechanism (ablation) | ComGS · N3DV flame steak | PSNR (dB): 32.82 -> 33.49 | Helped |
| Add adaptive motion-driven mechanism (ablation) | ComGS · N3DV flame salmon | PSNR (dB): 28.96 -> 29.32 | Helped |
| Add keypoint-driven motion modeling (ablation) | ComGS · N3DV flame steak | PSNR (dB): 31.26 -> 33.49 | Helped |
| Add keypoint-driven motion modeling (ablation) | ComGS · N3DV flame salmon | PSNR (dB): 27.75 -> 29.32 | Helped |
| Add error-aware correction (ablation) | ComGS · N3DV flame steak | PSNR (dB): 31.67 -> 33.49 | Helped |
| Add error-aware correction (ablation) | ComGS · N3DV flame salmon | PSNR (dB): 28.74 -> 29.32 | Helped |
| Replace KNN control with adaptive influence-field control (ablation) | ComGS · N3DV | PSNR (dB): 31.39 -> 31.87 | Helped |
Surprising findings
- Storage gains come with lower SSIM than some baselines on Flame Salmon (0.920 vs 0.952 for 4DGS), and ComGS-l does not beat QUEEN-l in PSNR on N3DV (32.12 vs 32.19).
Limitations
- Performance depends on the quality of the first-frame initialization, which propagates errors.
- Requires dense multi-view video inputs, which are expensive; not applicable to sparse-view or monocular settings.
- Training time in the encoding stage is not optimized (e.g., 37-45 s per sequence).
Experimental setup
Datasets: Neural 3D Video (N3DV) MeetRoom
Benchmarks: Neural 3D Video (N3DV) MeetRoom
Metrics: PSNR SSIM LPIPS storage (MB) training time (s) rendering FPS
Baselines: 3DGStream QUEEN-s QUEEN-l StreamRF TeTriRF TGH 4D-GS NeRFPlayer HyperReel SpaceTime I-NGP 3DG-S
Builds on: 3DGS 3DGStream QUEEN SC-GS KNN
Closest papers by meaning
- Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting (CVPR 2025 · similarity 0.94)
- HiCoM: Hierarchical Coherent Motion for Dynamic Streamable Scenes with 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
- 4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video (CVPR 2025 · similarity 0.92)
- Compressing Streamable Free-Viewpoint Videos to 0.1 MB per Frame (AAAI 2025 · similarity 0.91)
- 3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos (CVPR 2024 · similarity 0.91)
- SfM-Free 3D Gaussian Splatting via Hierarchical Training (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.