Research map / Image generation / Fast diffusion sampling
Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdaptiveDiffusion, a training-free, prompt-adaptive method that skips noise-prediction UNet calls per prompt using a third-order latent difference criterion, achieving about 2-5x speedup on image and video diffusion models.
Problem
Step-by-step noise prediction in diffusion models is computationally expensive, causing high latency for interactive use; existing acceleration uses fixed schedules for all prompts, though different prompts need different numbers of noise predictions.
Approach
Skip noise-prediction steps adaptively per prompt by reusing previous noise predictions, with the skip decision driven by the relative magnitude of the third-order latent difference, which is motivated by a one-step error bound and a greedy search for optimal skipping paths.
Key results
- On MS-COCO 2017 with SDXL and Euler scheduler, AdaptiveDiffusion achieves 2.01x speedup with LPIPS 0.168 and FID 6.11, versus DeepCache at 2.16x speedup with LPIPS 0.223 and FID 7.36.
- On ImageNet 256x256 with LDM-4, it reaches 5.56x speedup with PSNR 25.79 and LPIPS 0.090, compared with DeepCache at 6.35x and PSNR 24.98.
- On video generation, it reports PSNR 27.20 and FVD 274.5 on I2VGen-XL versus DeepCache's PSNR 20.82 and FVD 916.0, with speedup 1.93x versus 2.00x.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AdaptiveDiffusion instead of Deepcache | SD-1-5 with DDIM · MS-COCO 2017 | speedup ratio: 1.58$\times$ -> 1.77$\times$ | Helped |
| Use AdaptiveDiffusion instead of Adaptive DPM | SD-1-5 with DPM++ · MS-COCO 2017 | speedup ratio: 1.25$\times$ -> 1.57$\times$ | Helped |
| Use AdaptiveDiffusion instead of Deepcache | SDXL with DDIM · MS-COCO 2017 | speedup ratio: 1.75$\times$ -> 1.66$\times$ | Hurt |
| Use AdaptiveDiffusion instead of Deepcache | SDXL with Euler · MS-COCO 2017 | speedup ratio: 2.16$\times$ -> 2.01$\times$ | Hurt |
| Use AdaptiveDiffusion instead of Deepcache | LDM-4 · ImageNet 256×256 | speedup ratio: 6.35$\times$ -> 5.56$\times$ | Hurt |
| Increase skipping threshold δ from 0.005 to 0.02 (ablation) | AdaptiveDiffusion on SDXL · COCO2017, Euler | speedup ratio: 1.38$\times$ -> 3.06$\times$ | Helped |
| Increase skipping threshold δ from 0.005 to 0.02 (ablation) | AdaptiveDiffusion on SDXL · COCO2017, Euler | LPIPS: 0.023 -> 0.289 | Hurt |
| Increase maximum consecutive skipping steps Cmax from 4 to 10 (ablation) | AdaptiveDiffusion on SDXL · COCO2017, Euler | LPIPS: 0.168 -> 0.307 | Hurt |
| Use 25 rather than 50 original sampling steps (ablation) | AdaptiveDiffusion on SDXL · COCO2017 | speedup ratio: 2.01$\times$ -> 1.31$\times$ | Hurt |
| Use AdaptiveDiffusion instead of Deepcache | I2VGen-XL · Video generation | PSNR: 20.82 -> 27.20 | Helped |
| Use AdaptiveDiffusion instead of Deepcache | ModelScopeT2V · Video generation | FVD: 3383.0 -> 470.6 | Helped |
| Use AdaptiveDiffusion instead of Deepcache | SDXL with SDE-DPM · COCO2017 | speedup ratio: 1.63x -> 1.53x | Hurt |
Surprising findings
- Skipping noise-prediction steps while still updating latents at every step outperforms skipping full denoising steps (latent update plus noise prediction) used by prior methods.
- The method does not work well for extreme few-step generation (e.g., 10 steps) because latents change rapidly.
Limitations
- Only evaluated on image and video generation; not on 3D or speech.
- Limited benefit for extreme few-step sampling such as 10 steps.
- Hyperparameters (delta, C_max) must be tuned per model and sampling step count.
Experimental setup
Datasets: MS-COCO 2017 ImageNet AIGCBench MSR-VTT CIFAR10 LSUN
Benchmarks: MS-COCO 2017 ImageNet 256x256 AIGCBench MSR-VTT CIFAR10 LSUN
Metrics: PSNR LPIPS FID FVD MACs latency speedup ratio
Baselines: DeepCache Adaptive DPM-Solver
Builds on: DDIM DPM-Solver DeepCache
Closest papers by meaning
- AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models (NeurIPS 2025 · similarity 0.90)
- D^2-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models (AAAI 2025 · similarity 0.90)
- Soft Mixture Denoising: Beyond the Expressive Bottleneck of Diffusion Models (ICLR 2024 · similarity 0.89)
- EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.89)
- Interpreting and Improving Diffusion Models from an Optimization Perspective (ICML 2024 · similarity 0.89)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.