Research map / Image generation / Fast diffusion sampling
Lossy Compression with Pretrained Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Implements the DiffC pretrained-diffusion lossy compression algorithm end-to-end on Stable Diffusion 1.5/2.1/XL and Flux-dev, using a fast CUDA reverse-channel coder and a greedy timestep schedule, and compares it to HiFiC, MS-ILLM, DiffEIC and PerCo.
Problem
Pretrained diffusion models can in principle serve as lossy image compressors via DiffC, but reverse-channel coding was too slow and impractical, so DiffC had only been evaluated hypothetically and never fully implemented.
Approach
Makes DiffC practical by splitting high-KL steps into independent chunks of about 16 bits each, using a custom CUDA PFR kernel that avoids materializing samples, greedily optimizing the denoising timestep schedule via shortest path, and fixing per-step KL values in the protocol. It also adapts Flux (OT flow) to DDPM form.
Key results
- DiffC with Stable Diffusion 1.5 encodes and decodes images in under 10 seconds on an A40 GPU (encoding 0.6-9.4 s on Kodak).
- The custom CUDA RCC kernel is reported as about 64x faster than PyTorch/TensorFlow, encoding a 16-bit chunk in under 3 ms.
- DiffC achieves competitive rate-distortion curves vs HiFiC, MS-ILLM, DiffEIC and PerCo; it beats PerCo and DiffEIC on distortion, and beats HiFiC and MS-ILLM on perceptual quality, while Flux-based DiffC reaches 32.4 dB VAE-bound PSNR on Kodak.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use greedy-optimized timestep schedules rather than a fixed schedule (ablation) | DiffC (Stable Diffusion 1.5) · Kodak | rate-distortion | Helped |
| Reduce the denoising schedule from 163 steps to 54 steps (ablation) | DiffC (Stable Diffusion 1.5) · Kodak | reconstruction quality | Hurt |
| Use DiffC-F probability-flow denoising instead of DiffC-A ancestral sampling (ablation) | DiffC · Image compression | distortion | Helped |
| Replace DiffC-F with one-step DiffC-MSE denoising (ablation) | DiffC · Image compression | distortion | Hurt |
| Add prompt conditioning (ablation) | DiffC with Stable Diffusion 1.5/2.1 · Image compression | distortion | Hurt |
| Add prompt conditioning (ablation) | DiffC with Flux and SDXL · Image compression | visual quality (Q-Align) | Helped |
| Use SDXL refiner during reconstruction (ablation) | DiffC with SDXL · High signal-to-noise ratio domain | compression rate | No clear effect |
| Use SDXL refiner during reconstruction (ablation) | DiffC with SDXL · Image reconstruction | CLIP and quality scores | Helped |
| Replace ideal reverse-channel coding with practical reverse-channel coding (ablation) | DiffC · Rate-distortion | rate-distortion | Hurt |
| Use Stable Diffusion 1.5/2.1 rather than Flux | DiffC · Kodak | maximum reconstruction fidelity (PSNR): 25.7 vs. 32.4 | Hurt |
| Apply DiffC to pretrained diffusion models instead of using purpose-built generative compression methods | DiffC · Rate-distortion-perception comparison | rate-distortion | Mixed |
Surprising findings
- Hard-coding per-step KL values lower than the actual KL divergence results in surprisingly little distortion.
- Text prompts did not improve distortion metrics compared to unconditional DiffC, so the bits were better spent on the latent.
Limitations
- Encoding and decoding time for Flux and SDXL is very large (up to 321 s encoding for Flux), and the number of denoising steps remains a bottleneck.
- Performance is limited by the VAE's reconstruction fidelity for latent diffusion models; SD compression is only competitive at ultra-low bitrates.
- Performance degrades for image sizes outside the model's training distribution, and DiffC is not directly applicable to rectified flow models.
Experimental setup
Datasets: Kodak DIV2K
Benchmarks: Kodak DIV2K
Metrics: PSNR LPIPS CLIP score Q-Align SRCC on PIPAL
Baselines: HiFiC MS-ILLM DiffEIC PerCo Text+Sketch VTM
Builds on: DiffC Poisson Functional Representation DDIM MIRACLE
Closest papers by meaning
- OSCAR: One-Step Diffusion Codec Across Multiple Bit-rates (NeurIPS 2025 · similarity 0.90)
- Compressed Image Generation with Denoising Diffusion Codebook Models (ICML 2025 · similarity 0.89)
- Progressive Compression with Universally Quantized Diffusion Models (ICLR 2025 · similarity 0.89)
- One-Step Diffusion-Based Image Compression with Semantic Distillation (NeurIPS 2025 · similarity 0.88)
- Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion (ICML 2025 · similarity 0.88)
- Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.