Research map / Image generation / Image restoration
Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Conditional Latent Coding (CLC), which builds a universal image feature dictionary, synthesizes a conditioning latent from top-matched reference images, and uses it to guide end-to-end learned image compression.
Problem
Deep image compression struggles to exploit source correlation with external images, and prior reference-based methods use fixed or manually chosen side images rather than learned, dynamically synthesized references.
Approach
Construct a feature dictionary from web images (SPP + PCA + MiniBatch K-means), retrieve top-M reference images per input, and learn to match, align and fuse their latents (CLM and CLS modules) into a conditioning latent that conditions both encoder and decoder coding, with a theoretical perturbation bound.
Key results
- On Kodak, CLC with 3 reference images achieves -14.5% BD-rate (PSNR) vs VTM, and -13.9% on CLIC.
- Ablation: removing CLS raises BD-rate from -14.5% to -9.8% on Kodak; removing KV-cache increases encoding time from 1.05s to 1.87s.
- Abstract claims up to 1.2 dB gain with about 0.5% bpp overhead.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the number of reference images from 1 to 3 (ablation) | CLC · Kodak | BD-Rate_P: -10.2 -> -14.5 | Helped |
| Increase the number of reference images from 1 to 3 (ablation) | CLC · CLIC | BD-Rate_P: -9.8 -> -13.9 | Helped |
| Use 4 or 5 reference images instead of 3 (ablation) | CLC · Kodak and CLIC | BD-Rate_P: Kodak: -14.5 -> -14.3 (4), -14.2 (5); CLIC: -13.9 -> -13.7 (4), -13.6 (5) | Hurt |
| Increase dictionary cluster size from 1000 to 3000 (ablation) | CLC · Kodak | BD-Rate_P: -11.8 -> -14.5 | Helped |
| Increase dictionary cluster size from 3000 to 5000 (ablation) | CLC · Kodak | encoding time: 1.05 -> 5.67 | Hurt |
| Add CLM (ablation) | CLC · Kodak | BD-Rate_P: -12.3 -> -14.5 | Helped |
| Add CLS (ablation) | CLC · Kodak | BD-Rate_P: -9.8 -> -14.5 | Helped |
| Add KV-cache (ablation) | CLC · Kodak | encoding time: 1.87s -> 1.05s | Helped |
| Add KV-cache (ablation) | CLC · Kodak | BD-Rate_P: -14.4 -> -14.5 | No clear effect |
| Add multi-example query (ablation) | CLC · Kodak | BD-Rate_P: -13.8 -> -14.5 | Helped |
| Add multi-example query (ablation) | CLC · Kodak | BD-Rate_M: -14.6 -> -15.2 | Helped |
| Increase conditional-latent matching perturbation from ε = 0.1 to ε = 0.5 | CLC · Robustness experiment | Performance Reduction, PSNR: 3.7% -> 43.5% | Hurt |
Surprising findings
- Using more than three reference images degrades performance, attributed to redundancy.
- The theoretical error bound scales with 1/(1-rho), which diverges as reference correlation rho approaches 1, which is counter to the intuition that higher correlation helps.
Limitations
- Theoretical analysis relies on strong assumptions (spiked covariance, Gaussian-like noise) and its proof has informal steps.
- Robustness experiments only simulate random feature matching rather than real retrieval errors.
- Dictionary built from a fixed set of web images; generalization to other domains untested.
Experimental setup
Datasets: Flickr2W Flickr2K web-downloaded reference images
Benchmarks: Kodak CLIC
Metrics: PSNR MS-SSIM BD-rate bpp encoding time
Baselines: BPG VTM HM JPEG AV1 Hyperprior model Cheng et al. 2020 ELIC TCM
Builds on: hyperprior model slice-based autoregressive context model Transformer-CNN Mixture (TCM) Ball tree search
Closest papers by meaning
- Robustly overfitting latents for flexible neural image compression (NeurIPS 2024 · similarity 0.87)
- Generative Latent Coding for Ultra-Low Bitrate Image Compression (CVPR 2024 · similarity 0.86)
- Learned Image Compression with Dictionary-based Entropy Model (CVPR 2025 · similarity 0.85)
- BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities (ICLR 2025 · similarity 0.84)
- Test-time Adaptation for Image Compression with Distribution Regularization (ICLR 2025 · similarity 0.84)
- Robustness-Guided Image Synthesis for Data-Free Quantization (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.