Research map / Image generation / Fast diffusion sampling
Denoising Diffusion Step-aware Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DDSM, a slimmable diffusion UNet whose sub-network size is chosen per denoising step via NSGA-II evolutionary search, cutting FLOPs by 49-76% on image benchmarks without quality loss.
Problem
Diffusion models run the whole network at every denoising step, causing high inference cost; not all steps need equal compute.
Approach
Train a slimmable UNet supernet (sub-networks sampled at largest, smallest, and random widths) and use multi-objective evolutionary search to assign a different sub-network width to each diffusion step, with no retraining of pruned sub-networks; the step-aware scheme is compatible with DDIM and latent diffusion.
Key results
- On CIFAR-10, DDSM reaches FID 3.552 at 6.20 GFLOPs vs ADM 3.713 at 12.14 GFLOPs (49% savings).
- On ImageNet-64, DDSM achieves FID 24.714 at 19.40 GFLOPs vs ADM 27.000 at 82.06 GFLOPs (76% savings).
- On CelebA-HQ 64x64, DDSM reaches FID 6.039 at 19.58 GFLOPs vs ADM 6.755 at 49.88 GFLOPs (61% savings), and combining with DDIM-50 or LDM further reduces FLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply searched step-aware network widths | ADM · CIFAR-10 | FID: 3.713 -> 3.552 | Helped |
| apply searched step-aware network widths | ADM · CIFAR-10 | GFLOPs: 12.14 -> 6.20 | Helped |
| apply searched step-aware network widths | ADM · CelebA-HQ | FID: 6.755 -> 6.039 | Helped |
| apply searched step-aware network widths | ADM · ImageNet | FID: 27.000 -> 24.714 | Helped |
| apply searched step-aware network widths | ADM · LSUN-bedroom | FID: 5.027 -> 5.289 | Hurt |
| apply searched step-aware network widths | ADM · AFHQ | FID: 7.244 -> 6.249 | Helped |
| uniformly prune to mid width at every step (ablation) | ADM · CIFAR-10 | FID: 3.713 -> 4.697 | Hurt |
| uniformly prune to small width at every step (ablation) | ADM · CIFAR-10 | FID: 3.713 -> 5.936 | Hurt |
| use tiny width at every step (ablation) | ADM · CIFAR-10 | FID: 3.713 -> 18.638 | Hurt |
| choose model size randomly at each step (ablation) | ADM · CIFAR-10 | FID: 3.713 -> 7.727 | Hurt |
| use tiny model for steps [0, 250) instead of the large model (ablation) | DDPM-large · CIFAR-10 | FID: 3.71 -> 10.61 | Hurt |
| use tiny model for steps [500, 750) instead of the large model (ablation) | DDPM-large · CIFAR-10 | FID: 3.71 -> 3.30 | Helped |
Surprising findings
- Random per-step size strategies performed worse than uniform pruning to ADM-mid, showing that unguided step-wise mixing can hurt quality.
- Searched strategies differ by dataset: CelebA uses large models early and small late, while CIFAR-10 and ImageNet do the opposite.
Limitations
- Search and supernet training add substantial cost (e.g., 502 GPU hours training vs 278 for DDPM on CIFAR-10, plus 320 search hours).
- Step importance is dataset-dependent, so search must be repeated per dataset; no theoretical explanation provided.
- Some results slightly degrade quality (e.g., LSUN-bedroom FID 5.289 vs 5.027; CelebA-HQ 128 FID 7.71 vs 7.53).
Experimental setup
Datasets: CIFAR-10 CelebA-HQ LSUN-bedroom AFHQ ImageNet
Benchmarks: CIFAR-10 CelebA-HQ ImageNet LSUN-bedroom AFHQ
Metrics: FID GFLOPs GPU latency
Baselines: ADM DDIM Diff-pruning OMS-DPM LDM EDM
Builds on: DDPM ADM slimmable networks NSGA-II DDIM
Closest papers by meaning
- Multi-Step Denoising Scheduled Sampling: Towards Alleviating Exposure Bias for Diffusion Models (AAAI 2024 · similarity 0.92)
- Directly Denoising Diffusion Models (ICML 2024 · similarity 0.90)
- UDPM: Upsampling Diffusion Probabilistic Models (NeurIPS 2024 · similarity 0.90)
- Learning to Discretize Denoising Diffusion ODEs (ICLR 2025 · similarity 0.90)
- Interpreting and Improving Diffusion Models from an Optimization Perspective (ICML 2024 · similarity 0.90)
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.