Research map / Image generation / Fast diffusion sampling
DeepCache: Accelerating Diffusion Models for Free
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeepCache, a training-free method that caches U-Net high-level features across adjacent denoising steps and recomputes only shallow skip branches, speeding up DDPM, LDM and Stable Diffusion.
Problem
Diffusion model inference is slow due to sequential denoising and large model size; existing compression methods require costly retraining.
Approach
Exploit temporal redundancy of high-level U-Net features between adjacent denoising steps: cache the deep main-branch features at full-inference steps and reuse them while recomputing only shallow down/up blocks via skip connections, with a non-uniform 1:N schedule for caching intervals.
Key results
- On Stable Diffusion v1.5, DeepCache achieves 2.15x speedup on PartiPrompts with CLIP Score 29.46 vs 29.51 for 50-step PLMS, without retraining.
- On LDM-4-G ImageNet, uniform N=5 reaches 4.12x speedup with FID 3.59 vs 3.39 for the baseline 250-step DDIM.
- On CIFAR-10, DeepCache (N=2) gives 1.40x speedup with FID 4.35 vs 4.16 for the original DDPM and 5.29 for Diff-Pruning which required 100k retraining steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DeepCache with uniform N=5 caching instead of LDM-4-G | LDM-4-G · ImageNet 256 × 256 | FID: 3.60 -> 3.59 | No clear effect |
| Increase uniform caching interval from N=5 to N=10 (ablation) | DeepCache on LDM-4-G · ImageNet 256 × 256 | FID: 3.59 -> 4.41 | Hurt |
| Use non-uniform instead of uniform N=20 caching (ablation) | DeepCache on LDM-4-G · ImageNet 256 × 256 | FID: 8.23 -> 7.11 | Helped |
| Increase caching interval from N=2 to N=10 (ablation) | DeepCache on DDPM · CIFAR-10 32 × 32 | FID: 4.35 -> 9.74 | Hurt |
| Increase caching interval from N=2 to N=5 (ablation) | DeepCache on DDPM · LSUN-Bedroom 256 × 256 | FID: 6.69 -> 9.49 | Hurt |
| Replace cached features with a zero matrix (ablation) | DeepCache on DDPM · CIFAR10 | FID: 9.74 -> 192.98 | Hurt |
| Replace cached features with a zero matrix (ablation) | DeepCache on LDM-4-G · ImageNet | FID: 7.36 -> 312.12 | Hurt |
| Add shallow network inference to DDIM (ablation) | DDIM on DDPM · CIFAR-10 | FID: 4.67 -> 4.35 | Helped |
| Add shallow network inference to DDIM (ablation) | DDIM on DDPM · CIFAR-10 | FID: 13.36 -> 10.38 | Helped |
| Use DeepCache instead of PLMS - 50 steps | Stable Diffusion v1.5 · PartiPrompts | CLIP Score: 29.51 -> 29.46 | No clear effect |
| Use DeepCache instead of PLMS - 50 steps | Stable Diffusion v1.5 · COCO2017 | CLIP Score: 30.30 -> 30.23 | No clear effect |
Surprising findings
- Under the same throughput, DeepCache matches or slightly beats DDIM and PLMS on several settings.
- Replacing cached features with zeros raises FID from 9.74 to 192.98 on CIFAR-10, showing the cache is essential.
Limitations
- Speedup is constrained when the shallowest skip branch holds a large share of computation (e.g., 50%).
- Quality degrades noticeably at large caching intervals (e.g., N=20).
- Relies on the pre-defined structure of the pretrained U-Net, so it is tied to U-Net-based models.
Experimental setup
Datasets: CIFAR10 LSUN-Bedroom LSUN-Churches ImageNet MS-COCO 2017 PartiPrompts
Benchmarks: CIFAR10 LSUN-Bedroom LSUN-Churches ImageNet MS-COCO 2017 PartiPrompts
Metrics: FID sFID IS Precision Recall CLIP Score MACs throughput
Baselines: Diff-Pruning Spectral Diffusion BK-SDM DDIM PLMS
Builds on: DDPM DDIM PLMS Latent Diffusion Models Diff-Pruning
Closest papers by meaning
- CacheQuant: Comprehensively Accelerated Diffusion Models (CVPR 2025 · similarity 0.94)
- Cache Me if You Can: Accelerating Diffusion Models through Block Caching (CVPR 2024 · similarity 0.93)
- Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching (NeurIPS 2024 · similarity 0.91)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.90)
- FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality (ICLR 2025 · similarity 0.90)
- HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.