Research map / Image generation / Image restoration
Generative Latent Coding for Ultra-Low Bitrate Image Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Generative Latent Coding (GLC), which performs transform coding in the latent space of a generative VQ-VAE rather than pixel space, with a categorical hyper module and code-prediction-based auxiliary supervision, for ultra-low bitrate image compression.
Problem
Pixel-space transform coding struggles to achieve both high realism and high fidelity at ultra-low bitrates because pixel-space distortion does not align with human perception.
Approach
Encode images into the latent space of a generative VQ-VAE and perform learned transform coding there, with a categorical (codebook-based) hyper prior that captures semantic rather than low-level information, and code prediction used only as a training-time auxiliary supervision rather than in the inference pipeline.
Key results
- On CLIC 2020, GLC achieves the same FID as MS-ILLM with about 45% fewer bits.
- GLC achieves less than 0.04 bpp on natural images and less than 0.01 bpp on facial images while maintaining high visual quality.
- Categorical hyper module improves BD-Rate by 17.7% over factorized prior; code-prediction supervision gives 13.1% improvement; indices-map coding loses 66.2% vs transform coding.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace factorized prior with categorical prior for z (ablation) | GLC transform coding · CLIC 2020 test set | BD-Rate: 17.7% | Helped |
| replace transform coding with indices-map coding (ablation) | GLC latent compression · CLIC 2020 test set | BD-Rate: 66.2% | Hurt |
| add code-prediction-based supervision during training (ablation) | GLC · CLIC 2020 test set | BD-Rate: 13.1% | Helped |
| use code prediction in the network rather than as auxiliary training supervision (ablation) | GLC · CLIC 2020 test set | BD-Rate: 60.7% | Hurt |
| use GLC restoration application instead of cascading Restormer with GLC Codec | GLC image restoration · CLIC 2020 test set | FID: 10.79 -> 8.62 | Helped |
| use GLC restoration application instead of cascading Restormer with GLC Codec | GLC image restoration · CLIC 2020 test set | BPP: 0.0314 -> 0.0299 | Helped |
Surprising findings
- Placing code prediction inside the network before the decoder degrades performance by 60.7% BD-Rate, so it is used only as training-time supervision.
Limitations
- Generalization is limited as a generative codec trained on specified datasets.
- Cannot guarantee clear and accurate reconstruction of screen contents.
Experimental setup
Datasets: ImageNet OpenImages FFHQ CLIC 2020 CelebAHQ
Benchmarks: CLIC 2020 CelebAHQ Kodak DIV2K MS-COCO 30K
Metrics: bpp DISTS LPIPS FID KID BD-Rate PSNR MS-SSIM
Baselines: VVC TCM EVC FCC Text+Sketch HiFiC MS-ILLM
Builds on: VQ-VAE hyperprior entropy model (factorized prior) DCVC CodeFormer code prediction VQGAN
Closest papers by meaning
- HyperVQ: Enabling Hyperprior Entropy Modeling for VQ-Based Generative Image Compression (NeurIPS 2026 · similarity 0.88)
- Decouple Distortion from Perception: Region Adaptive Diffusion for Extreme-low Bitrate Perception Image Compression (CVPR 2025 · similarity 0.88)
- Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compression (CVPR 2025 · similarity 0.87)
- Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation (ICLR 2025 · similarity 0.87)
- Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion (ICML 2025 · similarity 0.87)
- BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.