Research map / Video / Video segmentation and tracking
4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 4DGC, a rate-aware 4D Gaussian compression framework for streamable free-viewpoint video that combines a multi-resolution motion grid, sparse compensated Gaussians, and jointly optimized implicit entropy models.
Problem
Storage and transmission of dynamic 3D Gaussian Splatting for streamable free-viewpoint video are expensive, and existing methods handle representation and compression separately, ignoring motion and the rate-distortion trade-off.
Approach
Model inter-frame motion of Gaussians with a compact multi-resolution motion grid plus sparsely added compensated Gaussians, and train the representation and a tiny implicit entropy model end-to-end with a rate-distortion loss using simulated quantization.
Key results
- On N3DV, 4DGC reaches 31.58 dB PSNR at 0.5 MB/frame versus 3DGStream's 31.54 dB at 8.1 MB/frame.
- BDBR reduction of 68.59% on N3DV relative to TeTriRF; 40.71% on MeetRoom and 59.99% on Google Immersive.
- Ablations show BD-PSNR drops of 1.86 dB (N3DV) when motion estimation is removed, 0.23 dB without Gaussian compensation, and 0.28 dB without joint optimization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add motion estimation (ablation) | 4DGC · N3DV | BD-PSNR: -1.86 dB | Helped |
| add motion estimation (ablation) | 4DGC · MeetRoom | BD-PSNR: -1.13 dB | Helped |
| add motion estimation (ablation) | 4DGC · Google Immersive | BD-PSNR: -1.62 dB | Helped |
| add Gaussian compensation (ablation) | 4DGC · N3DV | BD-PSNR: -0.23 dB | Helped |
| add Gaussian compensation (ablation) | 4DGC · MeetRoom | BD-PSNR: -0.22 dB | Helped |
| add Gaussian compensation (ablation) | 4DGC · Google Immersive | BD-PSNR: -0.09 dB | Helped |
| jointly optimize the representation and entropy models (ablation) | 4DGC · N3DV | BD-PSNR: -0.28 dB | Helped |
| jointly optimize the representation and entropy models (ablation) | 4DGC · MeetRoom | BD-PSNR: -0.34 dB | Helped |
| jointly optimize the representation and entropy models (ablation) | 4DGC · Google Immersive | BD-PSNR: -0.40 dB | Helped |
Limitations
- Relies on reconstruction quality of the first keyframe; poor initialization degrades overall performance.
- Requires multi-view video input and struggles with sparse-view reconstruction.
- Decoding is slower than rendering.
Experimental setup
Datasets: N3DV MeetRoom Google Immersive
Benchmarks: N3DV MeetRoom Google Immersive
Metrics: PSNR SSIM bitrate (MB/frame) FPS BD-PSNR BDBR training time encode/decode time
Baselines: K-Planes HyperReel MixVoxels NeRFPlayer StreamRF ReRF TeTriRF D-3DG 3DGStream
Builds on: 3D Gaussian Splatting 3DGStream implicit entropy model (Balle-style hyperprior-free entropy coding)
Closest papers by meaning
- Temporal Smoothness-Aware Rate-Distortion Optimized 4D Gaussian Splatting (NeurIPS 2025 · similarity 0.93)
- Compressing Streamable Free-Viewpoint Videos to 0.1 MB per Frame (AAAI 2025 · similarity 0.92)
- Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction (NeurIPS 2025 · similarity 0.92)
- 4DGCPro: Efficient Hierarchical 4D Gaussian Compression for Progressive Volumetric Video Streaming (NeurIPS 2025 · similarity 0.90)
- GIFStream: 4D Gaussian-based Immersive Video with Feature Stream (CVPR 2025 · similarity 0.90)
- Gaussian Splatting-based Volumetric Video Compression with Sparse 4D Anchors (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.