Research map / Video / Video segmentation and tracking
C3: High-Performance and Low-Complexity Neural Compression from a Single Image or Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces C3, an overfitted neural field codec (improving Cool-chic with soft-rounding, Kumaraswamy noise, and architecture changes, extended to video) that matches VTM on CLIC2020 images and VCT on UVG video at very low decoding MACs/pixel.
Problem
Neural codecs trained for generalization have high decoding complexity, while existing single-instance overfitted codecs (COIN, Cool-chic) have low complexity but poor rate-distortion performance, and no overfitted approach had been applied to video at competitive RD.
Approach
Overfit a small synthesis network and autoregressive entropy model per image or video, using improved quantization-aware optimization (soft-rounding with annealing, Kumaraswamy noise, smaller quantization step, soft-rounding gradients) and architecture changes (GELU, shifted log-scale, conditional entropy model), plus 3D latents, video patching, wide context with custom masking for video.
Key results
- On CLIC2020, C3 reduces BD-rate vs Cool-chic v2 by 20.8% and matches VTM (-0.1% BD-rate) with under 3k MACs/pixel decoding; adaptive C3 reaches -2.9% vs VTM.
- On UVG-1k, C3 matches VCT RD performance while requiring 4.4k MACs/pixel, less than 0.1% of VCT's decoding complexity.
- Ablations show soft-rounding, Kumaraswamy noise, and GELU activations contribute most of the gains (BD-rate increases of 22.2%, 3.9%, and 3.3% when removed).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Improve Cool-chic v2 with C3's optimization and architecture changes | Cool-chic v2 · CLIC2020 professional validation | BD-rate: −20.8% | Helped |
| Adapt the model per image | C3 · CLIC2020 professional validation | BD-rate vs VTM: −0.1% -> −2.9% | Helped |
| Add quantization step < 1 (ablation) | C3 · Kodak | BD-rate vs C3 Adaptive: 2.6% | Helped |
| Add adaptive learning rate in stage 2 (ablation) | C3 · Kodak | BD-rate vs C3 Adaptive: 3.4% | Helped |
| Add shifted log-scale (ablation) | C3 · Kodak | BD-rate vs C3 Adaptive: 4.2% | Helped |
| Use GELU activations (ablation) | C3 · Kodak | BD-rate vs C3 Adaptive: 12.6% | Helped |
| Add Kumaraswamy noise (ablation) | C3 · Kodak | BD-rate vs C3 Adaptive: 23.6% | Helped |
| Add soft-rounding (ablation) | C3 · Kodak | BD-rate vs C3 Adaptive: 39.8% | Helped |
| Add soft-rounding (ablation) | C3 · Kodak | BD-rate vs C3: 22.18% | Helped |
| Add Kumaraswamy noise (ablation) | C3 · Kodak | BD-rate vs C3: 3.90% | Helped |
| Use GELU activations (ablation) | C3 · Kodak | BD-rate vs C3: 3.27% | Helped |
| Add shifted log-scale (ablation) | C3 · Kodak | BD-rate vs C3: 0.87% | Helped |
Surprising findings
- C3 with a single fixed setting matches VTM on CLIC2020 despite a tiny overfitted decoder, outperforming Cool-chic v2 by 20.8% BD-rate.
- C3 is slightly worse than VTM on Kodak even though it beats VTM on CLIC2020 with adaptation.
Limitations
- Encoding is very slow, making it impractical for real-time encoding.
- Autoregressive entropy decoding is sequential and poorly suited to parallel hardware.
- Still behind state-of-the-art neural codecs such as MLIC+ and HiNeRV/MIMT in RD performance; slightly behind VTM on Kodak.
Experimental setup
Datasets: Kodak CLIC2020 UVG-1k
Benchmarks: Kodak CLIC2020 UVG-1k
Metrics: BD-rate PSNR decoding complexity (MACs/pixel) encoding and decoding time
Baselines: BPG VTM BMS CST EVC MLIC+ Cool-chic v2 HEVC HiNeRV FFNeRV VCT DCVC
Builds on: Cool-chic COIN NeRV BD-rate
Closest papers by meaning
- Towards Practical Real-Time Neural Video Compression (CVPR 2025 · similarity 0.88)
- Robustly overfitting latents for flexible neural image compression (NeurIPS 2024 · similarity 0.87)
- Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion (CVPR 2025 · similarity 0.86)
- Compress Clean Signal from Noisy Raw Image: A Self-Supervised Approach (ICML 2024 · similarity 0.86)
- Diversify, Contextualize, and Adapt: Efficient Entropy Modeling for Neural Image Codec (NeurIPS 2024 · similarity 0.86)
- Neural Video Compression with Feature Modulation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.