Research map / Video / Video segmentation and tracking
VRVVC: Variable-Rate NeRF-Based Volumetric Video Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VRVVC, an end-to-end jointly trained variable-rate compression framework for NeRF-based volumetric video that uses tri-plane residual modeling and a learnable-quantization, MLP-based entropy model controlled by predefined Lagrange multipliers, achieving multiple bitrates with a single model.
Problem
Storage and transmission of NeRF-based volumetric video are costly, and prior methods either optimize representation and compression separately or support only a single fixed rate-distortion tradeoff.
Approach
Models dynamic scenes as frame-by-frame tri-plane residuals within groups of frames, and compresses them with a variable-rate entropy model whose learnable quantization step is paired with predefined Lagrange multipliers, trained end-to-end with a multi-rate-distortion loss and a two-stage progressive schedule.
Key results
- On the DNA-Rendering dataset, VRVVC achieves BDBR of -81.86% (training views) and -83.51% (testing views) relative to TeTriRF.
- On the ReRF dataset, VRVVC achieves BDBR of -46.25% (training views) and -48.27% (testing views) relative to TeTriRF.
- A single model provides three bitrate operating points; Ours (Low) reaches 40 KB per frame on ReRF with 32.16 test PSNR, and 30 KB per frame on DNA-Rendering with 30.25 test PSNR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VRVVC instead of TeTriRF | VRVVC · ReRF Dataset, training view | BDBR: -46.25 | Helped |
| Use VRVVC instead of TeTriRF | VRVVC · ReRF Dataset, testing view | BDBR: -48.27 | Helped |
| Use VRVVC instead of TeTriRF | VRVVC · DNA-Rendering Dataset, training view | BDBR: -81.86 | Helped |
| Use VRVVC instead of TeTriRF | VRVVC · DNA-Rendering Dataset, testing view | BDBR: -83.51 | Helped |
| Add tri-plane residual dynamic modeling (ablation) | VRVVC · Volumetric video | bitrate | Helped |
| Add progressive training (ablation) | VRVVC · Volumetric video | bitrate | Helped |
| Jointly optimize the residual representation and entropy model (ablation) | VRVVC · Volumetric video | RD performance | Helped |
Limitations
- Evaluated only on two multi-view datasets (ReRF and DNA-Rendering) with two test views each
- Rendering-time claims are relative to a limited set of baselines; training uses a single V100 GPU
- Variable-rate operation relies on a predefined discrete set of eight Lagrange multipliers rather than continuous control
Experimental setup
Datasets: ReRF dataset DNA-Rendering dataset
Benchmarks: ReRF dataset DNA-Rendering dataset
Metrics: PSNR SSIM BDBR bitrate (KB per frame) rendering time
Baselines: K-Planes ReRF TeTriRF JointRF
Builds on: NeRF TeTriRF JointRF K-Planes ReRF
Closest papers by meaning
- Neural NeRF Compression (ICML 2024 · similarity 0.88)
- NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation (CVPR 2024 · similarity 0.88)
- TeTriRF: Temporal Tri-Plane Radiance Fields for Efficient Free-Viewpoint Video (CVPR 2024 · similarity 0.87)
- Neural Video Compression with Feature Modulation (CVPR 2024 · similarity 0.87)
- 4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video (CVPR 2025 · similarity 0.87)
- CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.