Research map / Image generation / Fast diffusion sampling
Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Tortoise and Hare Guidance (THG), a training-free sampler that splits the CFG diffusion ODE into a fine-grained "tortoise" noise-estimate ODE and a coarse-grid "hare" guidance ODE, cutting NFE by up to 30% on SD 1.5, SD 3.5 Large and AudioLDM 2.
Problem
Classifier-free guidance doubles the number of network evaluations per denoising step, making diffusion sampling costly; existing accelerators either reduce steps or reuse features but still spend two forward passes per step under CFG.
Approach
Reformulate the CFG diffusion ODE as a two-rate multirate system: integrate the noise estimate on the fine timestep grid and the additional guidance term on a coarser grid chosen by a Richardson-based error-ratio criterion, plus a guidance boost factor and a late-stage guidance cutoff.
Key results
- On SD 1.5 with DDIM, THG at 70 NFE reaches FID 14.165 and IR 0.11499 versus CFG at 100 NFE (FID 14.057, IR 0.14765), with the paper reporting up to 30% NFE reduction.
- On SD 1.5 with DPM-Solver-2, THG at 70 NFE gets FID 12.909 and IR 0.14926, beating CFG-Cache (FID 13.468, IR 0.11474) at the same NFE.
- On AudioLDM 2 at 70 NFE, THG improves FAD to 2.764 (vs 2.901 for CFG-Cache w/o FFT) and CLAP Score to 0.2342 (vs 0.2251).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use THG rather than CFG-Cache w/o FFT at 70 NFE | Stable Diffusion 1.5 with DDIM · COCO 2014 | ImageReward: 0.08757 -> 0.11499 | Helped |
| Use THG rather than CFG-Cache at 38 NFE | Stable Diffusion 3.5 Large with Euler · COCO 2014 | FID: 67.914 -> 68.252 | Hurt |
| Use THG rather than CFG-Cache at 38 NFE | Stable Diffusion 3.5 Large with Euler · COCO 2014 | CLIP Score: 26.668 -> 26.672 | Helped |
| Use THG rather than CFG-Cache w/o FFT at 70 NFE | AudioLDM 2 with DDIM · AudioCaps validation | FAD: 2.901 -> 2.764 | Helped |
| Increase guidance boost factor from b=1.00 to b=1.20 (ablation) | THG on Stable Diffusion 1.5 with DDIM · COCO 2014 | ImageReward: 0.09395 -> 0.13478 | Helped |
| Increase guidance boost factor from b=1.00 to b=1.20 (ablation) | THG on Stable Diffusion 1.5 with DDIM · COCO 2014 | FID: 13.811 -> 14.729 | Hurt |
| Lower error-ratio threshold from rho=1.1 to rho=0.9 (ablation) | THG on Stable Diffusion 1.5 with DDIM · COCO 2014 | FID: 14.232 -> 14.128 | Helped |
| Lower error-ratio threshold from rho=1.1 to rho=0.9 (ablation) | THG on Stable Diffusion 1.5 with DDIM · COCO 2014 | ImageReward: 0.11576 -> 0.11942 | Helped |
Surprising findings
- Selective Guidance attains the lowest FID and CMMD but very low (negative) ImageReward and CLIP Score, so distributional metrics can disagree with prompt fidelity.
- Vanilla CFG at 70 NFE has better FID (13.342) than THG at 70 NFE (14.232) in the appendix comparison, though THG is reported to have higher PSNR.
Limitations
- Experiments limited to latent diffusion models and a few benchmarks (COCO 2014, AudioCaps).
- Hyperparameters (rho, b, i_hi) are tuned per model and the coarse grid depends on omega, requiring calibration.
- Mixed results against baselines on distributional metrics: FID/CMMD sometimes worse than CFG-Cache on SD 3.5 Large.
Experimental setup
Datasets: COCO 2014 AudioCaps
Benchmarks: COCO 2014 AudioCaps
Metrics: FID CMMD CLIP Score ImageReward FAD CLAP Score PSNR
Baselines: CFG CFG-Cache CFG-Cache w/o FFT Selective Guidance Guidance Interval
Builds on: classifier-free guidance DDIM DPM-Solver multirate integration methods CFG-Cache
Closest papers by meaning
- Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models (ICLR 2025 · similarity 0.91)
- Token Perturbation Guidance for Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Multilevel and Sequential Monte Carlo for Training-Free Diffusion Guidance (NeurIPS 2026 · similarity 0.90)
- PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores (ICLR 2025 · similarity 0.90)
- Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models (NeurIPS 2024 · similarity 0.90)
- Tempered Guided Diffusion (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.