Research map / Video / Video segmentation and tracking
NVRC: Neural Video Representation Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NVRC, an INR-based video codec framework that compresses the neural representation itself with learned quantization, context-based and dual-axis entropy models, and hierarchical parameter coding, optimized end-to-end for rate-distortion.
Problem
INR-based video codecs still underperform standard codecs such as VVC VTM, partly because representation compression is simple and prior INR methods are not optimized end-to-end with rate-distortion including entropy model cost.
Approach
Treat the INR representation as the object to compress: per-block and per-axis learned quantization scales, a context-based Gaussian entropy model for feature grids, a dual-axis conditional Gaussian model for layer weights, and recursive hierarchical coding of the quantization and entropy-model parameters, all trained jointly with a rate-distortion loss using alternating R/D steps.
Key results
- NVRC achieves a 23.4% average BD-rate saving in PSNR over VVC VTM (Random Access) on UVG in RGB 4:4:4.
- NVRC achieves a 50.16% average BD-rate saving in PSNR over the original HiNeRV on UVG in RGB 4:4:4.
- Ablation: removing the rate loss in stage 1 causes up to a 35% BD-rate increase on challenging sequences (Jockey, ReadySetGo).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NVRC instead of VTM (RA) | Video compression · UVG RGB 4:4:4 | BD-rate, PSNR: -23.42% | Helped |
| Use NVRC instead of HiNeRV | Video compression · UVG RGB 4:4:4 | BD-rate, PSNR: -50.16% | Helped |
| Use NVRC instead of VTM (RA) | Video compression · MCL-JCV RGB 4:4:4 | BD-rate, PSNR: 36.91% | Hurt |
| Replace dual-axis layer Gaussian model with per-tensor conditional Gaussian model (ablation) | NVRC · UVG | BD-rate PSNR: 0.00% -> 13.04% | Hurt |
| Replace dual-axis layer Gaussian model with per-tensor conditional Gaussian model (ablation) | NVRC · UVG | BD-rate MS-SSIM: 0.00% -> 13.84% | Hurt |
| Replace grid context model with per-tensor conditional Gaussian model (ablation) | NVRC · UVG | BD-rate PSNR: 0.00% -> 11.06% | Hurt |
| Replace grid context model with per-tensor conditional Gaussian model (ablation) | NVRC · UVG | BD-rate MS-SSIM: 0.00% -> 10.24% | Hurt |
| Replace grid context and dual-axis layer Gaussian models with per-tensor conditional Gaussian models (ablation) | NVRC · UVG | BD-rate PSNR: 0.00% -> 23.37% | Hurt |
| Replace grid context and dual-axis layer Gaussian models with per-tensor conditional Gaussian models (ablation) | NVRC · UVG | BD-rate MS-SSIM: 0.00% -> 23.88% | Hurt |
| Store quantization and entropy model parameters in half-precision instead of entropy coding them (ablation) | NVRC · UVG | BD-rate PSNR: 0.00% -> 30.84% | Hurt |
| Store quantization and entropy model parameters in half-precision instead of entropy coding them (ablation) | NVRC · UVG | BD-rate MS-SSIM: 0.00% -> 30.07% | Hurt |
| Replace learned grid quantization steps with fixed steps (log-step size -4) (ablation) | NVRC · UVG | BD-rate PSNR: 0.00% -> 14.42% | Hurt |
Surprising findings
- On MCL-JCV, VTM (RA) is worse than DCVC-DC and HM is outperformed by NVRC only partially; VTM shows a positive BD-rate (+36.91% PSNR) against the default baseline there, indicating dataset-dependent ranking.
- Without alternating optimization the training step time rises by up to 40% but performance shows no noticeable difference.
Limitations
- High encoder complexity since each video requires content-specific overfitting.
- High latency because INR methods need all frames of a sequence during encoding.
- Floating-point operations limit reproducibility across devices.
Experimental setup
Datasets: UVG MCL-JCV JVET-CTC Class B
Benchmarks: UVG MCL-JCV JVET-CTC Class B
Metrics: BD-rate PSNR BD-rate MS-SSIM encoding/decoding FPS MACs
Baselines: VVC VTM (RA) HM-18.0 (RA) x265 veryslow DCVC-HEM DCVC-DC HiNeRV C3 HNeRV-Boost
Builds on: HiNeRV NeRV COOL-CHIC C3 Quant-Noise
Closest papers by meaning
- PNVC: Towards Practical INR-based Video Compression (AAAI 2025 · similarity 0.92)
- Neural Video Compression with Feature Modulation (CVPR 2024 · similarity 0.91)
- Towards Practical Real-Time Neural Video Compression (CVPR 2025 · similarity 0.91)
- Combining Frame and GOP Embeddings for Neural Video Representation (CVPR 2024 · similarity 0.90)
- On Quantizing Neural Representation for Variable-Rate Video Coding (ICLR 2025 · similarity 0.89)
- SINR: Sparsity Driven Compressed Implicit Neural Representations (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.