Research map / Image generation / Image restoration

Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion

ICML 2025 · Image generation › Image restoration · approach: Perceptual diffusion image compression

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ResULIC, an ultra-low-bitrate image compressor that combines MLLM-based semantic residual retrieval (text of what the latent decoder misses), prompt optimization against CLIP embeddings, and a compression-aware diffusion model that starts denoising from a noised compressed latent at a bitrate-dependent step.

Problem

Diffusion and MLLM-based image compressors at ultra-low bitrates (below 0.01 bpp) suffer from redundant text transmission and poor reconstruction fidelity/consistency, and their diffusion starting points are not aligned with the compression level.

Approach

Compute the semantic residual as the caption difference between original and latent-decoded images (only the missing information is transmitted as text, with index coding via CLIP vocabulary); optimize the text prompt with a CLIP-space projected embedding search guided by the diffusion loss (Pfo); and use a non-Markovian forward process that starts denoising from the compressed latent noised to a

Key results

• ResULIC achieves -80.7% BD-rate in LPIPS and -66.3% in FID versus PerCo on CLIC-2020 (ablation anchored to PerCo).
• Against MS-ILLM as anchor, ResULIC reaches -62.9% FID and -26.9% LPIPS BD-rate on CLIC-2020.
• With CDM, denoising drops to 4/3 steps and decoding takes 0.60s/0.43s on Kodak, with LPIPS BD-rate -54.4% versus the ANS baseline with 50/20 steps.

Measured effects reported in the paper

Surprising findings

• Deterministic sampling (eta=0) gives more consistent reconstructions than stochastic sampling (eta=1).

Limitations

• Pfo encoding adds about 175s per image, making encoding very slow.
• PerCo is evaluated via a reproduced version since the official implementation is not public.
• Relies on closed-source GPT-4o for captioning and residual extraction.

Experimental setup

Datasets: LSDIR Flickr2W CLIC-2020 Kodak DIV2K Tecnick MS-COCO-3K

Benchmarks: CLIC-2020 Kodak DIV2K Tecnick MS-COCO-3K

Metrics: BD-Rate LPIPS BD-Rate FID BD-Rate KID BD-Rate DISTS BD-Rate PSNR BD-Rate MS-SSIM

Baselines: VTM HiFiC MS-ILLM Text-Sketch DiffEIC CDC PerCo PASD ANS

Builds on: PerCo PEZ ControlNet-XS DDIM Stable Diffusion

Closest papers by meaning

• Towards image compression with perfect realism at ultra-low bitrates (ICLR 2024 · similarity 0.89)
• Decouple Distortion from Perception: Region Adaptive Diffusion for Extreme-low Bitrate Perception Image Compression (CVPR 2025 · similarity 0.89)
• One-Step Diffusion-Based Image Compression with Semantic Distillation (NeurIPS 2025 · similarity 0.89)
• Progressive Compression with Universally Quantized Diffusion Models (ICLR 2025 · similarity 0.89)
• Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compression (CVPR 2025 · similarity 0.88)
• OSCAR: One-Step Diffusion Codec Across Multiple Bit-rates (NeurIPS 2025 · similarity 0.88)