Research map / Video / Video segmentation and tracking
GIFStream: 4D Gaussian-based Immersive Video with Feature Stream
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GIFStream, a deformation-based 4D Gaussian representation with sparse time-dependent feature streams on Scaffold-GS-style anchors, compressed end-to-end (quantization-aware training, autoregressive entropy model) or via video codecs for immersive video.
Problem
Immersive 6-DoF dynamic video with 4D Gaussian Splatting needs a balance between rendering quality for fast motion and manageable storage; deformation-based methods lack capacity for fast motion while 4DGS methods are large and lack temporal correspondence for compression.
Approach
Augment a canonical-space deformation representation (anchor-based, Scaffold-GS) with motion-adaptive, sparse time-dependent feature streams that add capacity for fast motion and, when reorganized into 2D videos, provide temporal correspondence that enables efficient end-to-end and video-codec compression.
Key results
- On MPEG, GIFStream reaches 30.72 PSNR at 7 MB versus CSTG+PP 29.48 PSNR at 15 MB.
- On Panoptic Sports (basketball and boxes), GIFStream achieves 29.50 PSNR at 12.6 MB versus CSTG+PP 26.13 PSNR at 23.4 MB.
- On Neur3D, GIFStream achieves 31.75 PSNR at 10 MB, comparable to 4DGS (32.01 PSNR, 202 MB) and above deformation-based 4DGaussian (31.15 PSNR, 90 MB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GIFStream instead of STG | 4D scene representation · Neur3D | PSNR: 32.05 -> 31.75 | Hurt |
| use GIFStream instead of CSTG + PP | 4D scene representation · Neur3D | Storage (MB): 15 -> 10 | Helped |
| use GIFStream instead of 4DGS | 4D scene representation · Panoptic Sport (basketball and boxes) | PSNR: 28.68 -> 29.50 | Helped |
| use GIFStream instead of CSTG + PP | 4D scene representation · Panoptic Sport (basketball and boxes) | Storage (MB): 23.4 -> 12.6 | Helped |
| use GIFStream instead of 4DGS | 4D scene representation · MPEG | PSNR: 30.50 -> 30.72 | Helped |
| use GIFStream instead of CSTG + PP | 4D scene representation · MPEG | Storage (MB): 15 -> 7 | Helped |
| add time-dependent feature stream ft (ablation) | GIFStream · Bartender | PSNR: 30.59 -> 31.94 | Helped |
| add sparse mask Mde (ablation) | GIFStream · Bartender | Storage (MB): 6.5 -> 5.3 | Helped |
| add KNN feature aggregation for motion prediction (ablation) | GIFStream · Bartender | PSNR: 31.86 -> 31.94 | Helped |
| use GIFStream densification instead of Scaffold-GS densification (ablation) | GIFStream · Bartender | PSNR: 31.82 -> 31.94 | Helped |
| add compression (quantization-aware training, entropy regularization, and entropy coding) (ablation) | GIFStream · Bartender | Storage (MB): 46.1 -> 5.3 | Helped |
| add compression (quantization-aware training, entropy regularization, and entropy coding) (ablation) | GIFStream · Bartender | PSNR: 32.13 -> 31.94 | Hurt |
Surprising findings
- Time-dependent feature streams are highly sparse: about 30% of anchors in challenging scenes and about 0.3% in simpler scenes need them.
Limitations
- Rendering FPS is lower than 4DGS methods due to deformation computation.
- Feature streams add parameters, and storage depends on how many anchors need time-dependent features.
Experimental setup
Datasets: Neur3D Panoptic Sports MPEG dataset
Benchmarks: Neur3D Panoptic Sports MPEG dataset
Metrics: PSNR SSIM LPIPS FPS storage (MB) bitrate
Baselines: CSTG STG 4DGS E-D3DGS 4DGaussian NeRFPlayer K-Planes Dynamic3DGS
Builds on: Scaffold-GS CSTG HAC Compact 3DGS
Closest papers by meaning
- 4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video (CVPR 2025 · similarity 0.90)
- Temporal Smoothness-Aware Rate-Distortion Optimized 4D Gaussian Splatting (NeurIPS 2025 · similarity 0.89)
- Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction (NeurIPS 2025 · similarity 0.88)
- 4DGCPro: Efficient Hierarchical 4D Gaussian Compression for Progressive Volumetric Video Streaming (NeurIPS 2025 · similarity 0.88)
- Compressing Streamable Free-Viewpoint Videos to 0.1 MB per Frame (AAAI 2025 · similarity 0.88)
- Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.