Research map / Video / Video segmentation and tracking
Neural Video Compression with Context Modulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DCMVC, a conditional-coding neural video codec that modulates propagated temporal context with a flow-oriented context from the reference frame via global-local context compensation and a decoupling loss.
Problem
Conditional-coding neural video codecs rely on propagated reference features that carry irrelevant information and accumulate errors over long prediction chains, limiting how well the reference frame is exploited for temporal context.
Approach
Generate an additional oriented temporal context from the reference frame using an oriented flow (SpyNet-based inter-frame correlation extractor), then modulate the propagated context with it through a global-local context compensation network supervised by a decoupling loss that makes global features correlated and local features decorrelated.
Key results
- DCMVC achieves 22.7% average BD-rate saving over VTM-13.2 under intra-period -1 (RGB-PSNR).
- DCMVC achieves 10.1% average bitrate saving over DCVC-FM on the averaged comparison in the abstract; under intra-period 32 it saves 19.4% vs VTM and 9.9% vs DCVC-FM in Table 1 with intra-period -1 saving 12.6% vs DCVC-FM.
- Ablation: full model Mg with 32-frame training and context modulation achieves 10.3% BD-rate saving over the reproduced DCVC-DC baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace DCVC-DC with DCMVC | Video compression codec · Average, intra-period 32 | BD-Rate (%): –14.1 -> –19.4 | Helped |
| Replace DCVC-FM with DCMVC | Video compression codec · Average, intra-period -1 | BD-Rate (%): –12.6 -> -22.7 | Helped |
| Add flow orientation (ablation) | DCVC-DC baseline · HEVC datasets, intra-period 32 | BD-Rate (%): 0.0 -> -1.9 | Helped |
| Use context compensation with oriented context extracted directly from the prediction frame (ablation) | DCVC-DC baseline · HEVC datasets, intra-period 32 | BD-Rate (%): 0.0 -> -3.5 | Helped |
| Add flow orientation and context compensation (ablation) | DCVC-DC baseline · HEVC datasets, intra-period 32 | BD-Rate (%): 0.0 -> -4.4 | Helped |
| Add decoupling loss (ablation) | DCMVC with flow orientation and context compensation · HEVC datasets, intra-period 32 | BD-Rate (%): -4.4 -> -5.4 | Helped |
| Train on longer sequences (32 frames) (ablation) | DCVC-DC baseline · HEVC datasets, intra-period 32 | BD-Rate (%): 0.0 -> -4.3 | Helped |
| Add context modulation and long-sequence training (ablation) | DCVC-DC baseline · HEVC datasets, intra-period 32 | BD-Rate (%): 0.0 -> -10.3 | Helped |
Surprising findings
- Naively concatenating the oriented and propagated contexts (Mb) gives only 1.9% saving, and a variant that extracts oriented context directly from the prediction frame (Mc) gives 3.5%, showing the flow-orientation step matters.
Limitations
- Uses rigid warp operation for temporal alignment.
- Motion vectors are only constrained indirectly through the RD loss, which may be suboptimal.
- Higher MACs and decoding time than DCVC-DC and DCVC-FM.
Experimental setup
Datasets: Vimeo-90k Vimeo raw videos (9000-sequence subset)
Benchmarks: UVG MCL-JCV USTC-TD HEVC Class B HEVC Class C HEVC Class D HEVC Class E
Metrics: BD-rate PSNR bits per pixel MACs parameters encoding time decoding time
Baselines: VTM-13.2 DCVC DCVC-TCM DCVC-HEM SDD DCVC-DC DCVC-FM
Builds on: DCVC-DC DCVC-FM SpyNet DCVC-TCM
Closest papers by meaning
- Towards Practical Real-Time Neural Video Compression (CVPR 2025 · similarity 0.90)
- Neural Video Compression with Feature Modulation (CVPR 2024 · similarity 0.90)
- Neural B-frame Video Compression with Bi-directional Reference Harmonization (NeurIPS 2025 · similarity 0.89)
- Augmented Deep Contexts for Spatially Embedded Video Coding (CVPR 2025 · similarity 0.88)
- ECVC: Exploiting Non-Local Correlations in Multiple Frames for Contextual Video Compression (CVPR 2025 · similarity 0.88)
- DeNC: Unleash Neural Codecs in Video Streaming with Diffusion Enhancement (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.