Research map / 3D vision / Gaussian splatting
QUEEN: QUantized Efficient ENcoding of Dynamic Gaussians for Streaming Free-viewpoint Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QUEEN, a framework that streams free-viewpoint video by learning quantized, sparsified 3D Gaussian attribute residuals between consecutive frames, guided by viewspace gradient differences.
Problem
Online streaming of free-viewpoint video requires on-the-fly updates of a volumetric representation with fast training and rendering and a small per-frame memory footprint for transmission.
Approach
Learn residuals for all 3D-GS attributes between consecutive frames without structural constraints; quantize non-position residuals with a learned shared linear latent decoder and entropy coding; sparsify position residuals with a hard concrete gate; initialize gates and mask training regions using viewspace gradient differences between frames.
Key results
- On N3DV, QUEEN-l reaches 32.19 dB PSNR with 0.75 MB per frame storage and 7.9 s training, versus 3DGStream* at 31.58 dB, 7.80 MB, 8.5 s.
- On Immersive, QUEEN-l achieves 29.22 dB PSNR, 1.79 MB storage and 19.7 s training, versus 3DGStream* at 25.18 dB and 8.83 MB.
- Some highly dynamic scenes reach about 0.7 MB per frame with training under 5 s and rendering around 350 FPS (QUEEN-s: 345 FPS on N3DV).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use QUEEN-l instead of 3DGStream* | Online FVV · N3DV | PSNR (dB): 31.58 -> 32.19 | Helped |
| Use QUEEN-l instead of 3DGStream* | Online FVV · N3DV | Storage (MB): 7.80 -> 0.75 | Helped |
| Use QUEEN-l instead of 3DGStream* | Online FVV · N3DV | Rendering (FPS): 261 -> 248 | Hurt |
| Use QUEEN-l instead of 3DGStream* | Online FVV · Immersive | PSNR (dB): +4 dB | Helped |
| Use QUEEN-l instead of 3DGStream* | Online FVV · Immersive | Training (sec): 32.4 -> 19.7 | Helped |
| Add attribute quantization except for position (ablation) | QUEEN · N3DV | Storage (MB): 44.36 -> 4.18 | Helped |
| Add attribute quantization except for position (ablation) | QUEEN · N3DV | PSNR (dB): 31.66 -> 32.04 | Helped |
| Add position-residual gating (ablation) | QUEEN · N3DV | Storage (MB): 4.18 -> 0.72 | Helped |
| Add viewspace-gradient-difference gate initialization (ablation) | QUEEN · N3DV | PSNR (dB): 32.05 -> 32.14 | Helped |
| Add masked training (ablation) | QUEEN · Immersive | Training (sec): 27.81 -> 19.70 | Helped |
| Update appearance attributes per frame (ablation) | QUEEN · N3DV | PSNR (dB): 31.61 -> 32.03 | Helped |
| Quantize the scaling attribute (ablation) | QUEEN · N3DV | PSNR (dB): 31.69 -> 32.08 | Helped |
Surprising findings
- Quantizing the scaling attribute improved PSNR over the unquantized baseline (e.g., 31.69 to 32.08 dB), not just reducing size.
- Position residuals are sensitive to quantization, so gating (sparsity) outperformed quantization for position at similar sizes.
Limitations
- Authors note future work is needed for sparse-view reconstruction and long-duration sequences.
- Masked training is applied only for a fraction of iterations because updating masked Gaussians can change unmasked pixels.
Experimental setup
Datasets: Neural 3D Videos (N3DV) Immersive Videos (Google)
Benchmarks: Neural 3D Videos (N3DV) Immersive Videos
Metrics: PSNR SSIM LPIPS (VGG) storage (MB) training time (sec) rendering FPS
Baselines: 3DGStream TeTriRF StreamRF NeRFPlayer HyperReel SpaceTime
Builds on: 3D Gaussian Splatting 3DGStream hard concrete gate straight-through estimator
Closest papers by meaning
- Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction (NeurIPS 2025 · similarity 0.89)
- 3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos (CVPR 2024 · similarity 0.89)
- 4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video (CVPR 2025 · similarity 0.89)
- Compressing Streamable Free-Viewpoint Videos to 0.1 MB per Frame (AAAI 2025 · similarity 0.88)
- Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting (CVPR 2025 · similarity 0.87)
- L4GM: Large 4D Gaussian Reconstruction Model (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.