Research map / Image generation / Fast diffusion sampling
Compressed Image Generation with Denoising Diffusion Codebook Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Replaces Gaussian noise in DDPM reverse sampling with indices into fixed random Gaussian codebooks, yielding images that are losslessly represented by index bitstreams, and applies this to perceptual image compression and compressed conditional generation/restoration.
Problem
Generative diffusion models sample from an unbounded continuous Gaussian noise space, which is inefficient for producing samples together with compact bit-stream representations for compression and conditional generation tasks.
Approach
Pre-sample fixed iid Gaussian codebooks per timestep and let the reverse diffusion pick a codebook index at each step (random for generation, or chosen by matching the residual to the target for compression); the index sequence is a lossless bitstream, and the same selection rule extends to conditional generation and restoration.
Key results
- DDCM with K=64 achieves FID similar to DDPM for class-conditional ImageNet and SD 2.1 generation, suggesting the continuous noise space is redundant.
- On Kodak24 at ~0.095 BPP, the method reports FID 24.2 versus 35.3 for PSC-P and 26.4 for PerCo (SD) at 0.127 BPP, with better PSNR than perceptual baselines at matched rates.
- For 4x super-resolution and colorization with K=4096 (~0.183 BPP), the method achieves better FID than DPS and DDNM, while PSNR is slightly lower.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DDCM with K=64 instead of continuous Gaussian noise sampling (ablation) | DDPM generation · ImageNet 256×256 and MS-COCO prompts with Stable Diffusion 2.1 | FID | No clear effect |
| Use smaller codebooks, down to K=16, instead of continuous Gaussian noise sampling (ablation) | DDPM generation · ImageNet 256×256 and Stable Diffusion 2.1 generation | generative performance | No clear effect |
| Add BLIP-2 text prompts to the otherwise unconditional compression method (ablation) | DDCM image compression with Stable Diffusion 2.1 · 512×512 image compression | compression quality | Hurt |
| Use timestep-adapted codebook sizes instead of naive timestep skipping (ablation) | DDCM latent-space image compression · 768×768 image compression | FID | Helped |
| Use timestep-adapted codebook sizes instead of naive timestep skipping (ablation) | DDCM latent-space image compression · 768×768 image compression | PSNR | Hurt |
| Increase the codebook size K (ablation) | DDCM posterior sampling · Compressed conditional generation | posterior sampling accuracy | Helped |
| Compress outputs of DPS or DDNM after generating them, rather than generate them in compressed form with DDCM (ablation) | DPS and DDNM image restoration · ImageNet colorization and 4× super-resolution | FID | Hurt |
| Generate restoration outputs directly in compressed form using DDCM instead of DPS or DDNM | Unconditional ImageNet DDM · ImageNet colorization and 4× super-resolution | FID | Helped |
| Generate restoration outputs directly in compressed form using DDCM instead of DPS or DDNM | Unconditional ImageNet DDM · ImageNet colorization and 4× super-resolution | PSNR | Hurt |
| Use DDCM latent-space compression at the highest bit rates | Stable Diffusion 2.1 image compression · Image compression | distortion | Hurt |
| Optimize a no-reference image quality measure while selecting restoration indices | DDCM face restoration · WIDER-Test | perceptual quality | Helped |
| Optimize a no-reference image quality measure while selecting restoration indices | DDCM face restoration · WIDER-Test | distortion | Hurt |
Surprising findings
- Even very small codebooks (K=16 to 64) retain sample quality and diversity comparable to continuous-noise DDPM.
- Compressing outputs of DPS and DDNM with the same codec harms either FID or PSNR, so naive compression does not match the proposed compressed conditional generation.
Limitations
- Latent-space variant is bounded in distortion by the SD 2.1 VAE encoder-decoder, underperforming at high bit rates.
- No entropy coding of selected indices, so bit rates could be reduced further; higher rates need matching pursuit which is heuristic.
- Lacks theoretical explanation of why small codebooks suffice; text prompts hurt compression performance.
Experimental setup
Datasets: ImageNet 256x256 MS-COCO Kodak24 DIV2K CLIC2020 CelebA-Test LFW-Test WebPhoto-Test WIDER-Test FFHQ
Benchmarks: ImageNet 256x256 Kodak24 CLIC2020 DIV2K CelebA-Test LFW-Test WebPhoto-Test WIDER-Test
Metrics: FID PSNR LPIPS KID FD DINOv2 NIQE CLIP-IQA+ TOPIQ ProxPSNR BPP
Baselines: BPG HiFiC IPIC PSC PerCo (SD) ILLM CRDR DPS DDNM PMRF DifFace BFRffusion
Builds on: DDPM classifier guidance Stable Diffusion matching pursuit DPS DDNM
Closest papers by meaning
- OSCAR: One-Step Diffusion Codec Across Multiple Bit-rates (NeurIPS 2025 · similarity 0.91)
- Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compression (CVPR 2025 · similarity 0.90)
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.90)
- Progressive Compression with Universally Quantized Diffusion Models (ICLR 2025 · similarity 0.90)
- Diffusion Models Without Attention (CVPR 2024 · similarity 0.89)
- Residual Denoising Diffusion Models (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.