Research map / Image generation / Fast diffusion sampling
No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes independent condition guidance (ICG) to get classifier-free guidance from a purely conditional diffusion model by feeding an independent random condition, and time-step guidance (TSG) that perturbs the time-step embedding to give CFG-like gains for conditional and unconditional models.
Problem
Classifier-free guidance requires training an unconditional branch (via condition dropping or a separate model), which complicates training of conditional diffusion models and provides no clear extension to unconditional generation.
Approach
Replacing the condition with an input-independent random condition makes the conditional score reduce to the unconditional score, so CFG can be simulated at inference without auxiliary training (ICG). TSG extends the guidance idea to any diffusion model by contrasting outputs for clean and noise-perturbed time-step embeddings, motivated by a first-order Langevin-like analysis.
Key results
- On DiT-XL/2, ICG with random condition reaches FID 5.50 at precision 0.83, comparable to CFG-based guidance.
- Combining ICG and TSG on DiT-XL/2 reaches FID 5.76 vs 6.47 for ICG alone and 15.49 for unguided sampling.
- Training a purely conditional DiT without label dropping gives about 20% lower FID or about 30% faster convergence at equal iterations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine ICG and TSG instead of using either alone | DiT-XL/2 generation · DiT-XL/2 generation | FID: 6.47 and 9.55 -> 5.76 | Helped |
| Add ICG | unguided DiT-XL/2 sampling · DiT-XL/2 generation | FID: 15.49 -> 6.47 | Helped |
| Add TSG | unguided DiT-XL/2 sampling · DiT-XL/2 generation | FID: 15.49 -> 9.55 | Helped |
| Combine ICG and TSG rather than use unguided sampling | DiT-XL/2 generation · DiT-XL/2 generation | recall: 0.74 -> 0.65 | Hurt |
| Use Gaussian noise instead of a random condition in ICG (ablation) | DiT-XL/2 ICG · DiT-XL/2 generation | FID: 5.55 -> 5.50 | No clear effect |
| Add CADS to ICG | DiT-XL/2 ICG · DiT-XL/2 generation | recall: 0.32 -> 0.61 | Helped |
| Add CADS to ICG | DiT-XL/2 ICG · DiT-XL/2 generation | precision: 0.89 -> 0.78 | Hurt |
| Remove label dropping and train a purely conditional DiT, then guide with ICG | DiT class-conditional ImageNet training · ImageNet class-conditional generation | FID: around a 20% reduction in FID | Helped |
| Reallocate training resources from the CFG objective to conditional training | DiT class-conditional ImageNet training · ImageNet class-conditional generation | training convergence: approximately 30% | Helped |
| Increase perturbation of the time-step embedding (ablation) | TSG · DiT-XL/2 generation | precision | Helped |
| Increase perturbation of the time-step embedding (ablation) | TSG · DiT-XL/2 generation | recall | Hurt |
| Add too much noise to the time-step embedding (ablation) | TSG · DiT-XL/2 generation | image quality | Hurt |
Surprising findings
- Training without the CFG label-dropping objective yields better models than standard label-dropping training when ICG is used at inference.
Limitations
- Guided sampling still requires two network evaluations per step, the same cost as CFG.
- TSG hyperparameters (scale, power, layer range) were not exhaustively tuned and need per-model settings.
- Approximation error in ICG for out-of-distribution independent conditions is bounded only through a KL-based analysis.
Experimental setup
Datasets: ImageNet MS COCO 2017
Benchmarks: ImageNet MS COCO 2017
Metrics: FID precision recall FD_DINOv2
Baselines: CFG SAG PAG unguided sampling CADS
Builds on: classifier-free guidance classifier guidance EDM ControlNet CADS
Closest papers by meaning
- TFG: Unified Training-Free Guidance for Diffusion Models (NeurIPS 2024 · similarity 0.94)
- Token Perturbation Guidance for Diffusion Models (NeurIPS 2025 · similarity 0.93)
- Rectified Diffusion Guidance for Conditional Generation (CVPR 2025 · similarity 0.93)
- Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models (ICLR 2025 · similarity 0.92)
- Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models (AAAI 2025 · similarity 0.92)
- Elucidating the design space of classifier-guided diffusion generation (ICLR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.