Research map / Image generation / Fast diffusion sampling
Guiding a Diffusion Model with a Bad Version of Itself
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes autoguidance, which guides a diffusion model with a smaller, less-trained version of itself instead of an unconditional model, improving ImageNet FID records at 64 and 512 resolution.
Problem
Classifier-free guidance improves image quality but entangles it with reduced variation and requires an unconditional model that differs in task from the conditional one, causing skewed compositions and limiting control.
Approach
Guide the main diffusion model with an inferior version of itself (same task, conditioning and data, but lower capacity and/or less training), so quality improvement is separated from prompt alignment and variation is preserved.
Key results
- Autoguidance with EDM2-S improves ImageNet-512 FID from the EDM2-S no-guidance baseline to 1.34 (best reported for that model size), beating CFG + guidance interval.
- Using EDM2-XXL with an M-sized guiding model sets record ImageNet-512 FID of 1.25 (per abstract).
- On ImageNet-64, autoguidance sets a record FID of 1.01 (per abstract), and also applies to unconditional EDM2-S, reducing its FID substantially.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Guide EDM2-S with a smaller, less-trained conditional model | EDM2-S · ImageNet-512 | FID | Helped |
| Use autoguidance instead of classifier-free guidance | EDM2-S · ImageNet-512 | FID | Helped |
| Guide an unconditional EDM2-S model with a smaller, less-trained unconditional model | Unconditional EDM2-S · ImageNet-512 unconditional | FID | Helped |
| Allow independent EMA lengths for the main and guiding models (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | Helped |
| Use a same-capacity guiding model trained for less time, rather than reducing both capacity and training time (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | Hurt |
| Train the reduced-capacity guiding model as long as the main model, rather than training it for less time (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | Hurt |
| Further halve the guiding model's training budget from the best setting (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | No clear effect |
| Double the guiding model's training budget from the best setting (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | Hurt |
| Reduce guiding-model capacity by two model sizes instead of one (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | Mixed |
| Reduce the amount of training data used for the guiding model (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | No clear effect |
| Add guidance interval on top of autoguidance (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | Hurt |
| Use synthetic degradations to derive the guiding model from the main model (ablation) | EDM2-S with autoguidance · ImageNet-512 | FID | Hurt |
Surprising findings
- Guiding with a degraded copy created by synthetic corruptions (dropout, input noise) of the main model does not help unless the corruptions match, suggesting the guiding model must share the main model's real degradations.
- Applying guidance interval on top of autoguidance reduces its benefit.
Limitations
- Requires training a separate guiding model, adding cost and needing a suitable early snapshot or smaller model.
- Hyperparameters (guidance weight, EMA lengths, guiding model capacity and training time) require grid search per configuration.
- Not applicable as-is to large-scale generators where earlier snapshots are unavailable; theoretical conditions for benefit are not formally proven.
Experimental setup
Datasets: ImageNet-512 ImageNet-64 ILSVRC2012 2D toy fractal dataset
Benchmarks: ImageNet-512 ImageNet-64 DeepFloyd IF prompt samples
Metrics: FID FD_DINOv2
Baselines: classifier-free guidance CFG + guidance interval EDM2-S EDM2-XXL RIN DeepFloyd IF
Builds on: classifier-free guidance EDM2 EDM post-hoc EMA guidance interval
Closest papers by meaning
- Elucidating the design space of classifier-guided diffusion generation (ICLR 2024 · similarity 0.90)
- REG: Rectified Gradient Guidance for Conditional Diffusion Models (ICML 2025 · similarity 0.90)
- Plug-and-Play Diffusion Distillation (CVPR 2024 · similarity 0.90)
- Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Rectified Diffusion Guidance for Conditional Generation (CVPR 2025 · similarity 0.89)
- Understanding and Improving Training-free Loss-based Diffusion Guidance (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.