Research map / Image generation / Fast diffusion sampling
TADA: Improved Diffusion Sampling with Training-free Augmented DynAmics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TADA, a training-free ODE sampler that reuses pretrained single-input diffusion models by feeding a reweighted combination of higher-dimensional momentum-system noise variables, and proves training equivalence of momentum diffusion models with conventional ones.
Problem
Diffusion model sampling is slow and ODE samplers often yield lower fidelity than SDE samplers at low NFE; momentum diffusion models were thought to need retraining and their training benefits were unclear.
Approach
Show that momentum diffusion training reduces to a conventional diffusion objective on an r-weighted linear combination of the N augmented variables, so pretrained single-input models can be reused; sample with higher-dimensional initial noise under a momentum ODE, which induces SDE-like diversity via pseudo-noise while remaining deterministic.
Key results
- On ImageNet-512 with EDM2, TADA reaches comparable FID to the best baseline up to 186% faster (abstract claim).
- TADA consistently outperforms UniPC and DPM-Solver++ across tested NFEs on ImageNet-64 (EDM) and ImageNet-512 (EDM2).
- On SD3, TADA outperforms baselines on HPSv2 across CFG strengths, with gap narrowing as CFG increases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TADA sampling instead of conventional ODE and SDE solvers | Pretrained EDM and EDM2 models · ImageNet-64 and ImageNet-512 | FID | Helped |
| Use TADA sampling instead of UniPC, Flow Matching, or Stochastic Adam | Stable Diffusion 3 · HPSv2 | HPSv2 score | Helped |
| Increase the augmented variable count to N>2 (ablation) | TADA · ImageNet-64 | FID | Helped |
| Increase the augmented variable count to N>2 (ablation) | TADA · ImageNet-512 | FID | No clear effect |
| Increase the standard deviation of the final augmented variable (ablation) | TADA · Image generation from the same initial condition | sample diversity | Helped |
| Add noise or noisy-data augmentation to conventional diffusion training | Diffusion-model training · Training objective | training benefit | No clear effect |
Surprising findings
- Theoretically, momentum diffusion training gives no training benefit over conventional diffusion; the benefit appears only in sampling.
- Increasing the variance of the last augmented variable did not consistently improve ImageNet-512 results.
Limitations
- Uses only a basic exponential integrator; more advanced solvers like UniPC and DPM-Solver++ not yet integrated.
- Falls short of SDE results with 511 NFEs; limited control over injected stochasticity due to naive choice of A_t.
- Performance gains diminish as model capacity and CFG strength increase.
Experimental setup
Datasets: ImageNet-64 ImageNet-512 HPSv2 prompts
Benchmarks: ImageNet-64 ImageNet-512 HPSv2
Metrics: FID FD-DINOv2 HPSv2 score
Baselines: UniPC DPM-Solver++ Stochastic Adams Solver Flow Matching default solver in SD3
Builds on: AGM CLD DPM-Solver++ UniPC Denoising Diffusion Implicit Models
Closest papers by meaning
- PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores (ICLR 2025 · similarity 0.91)
- Accelerating Diffusion Sampling with Optimized Time Steps (CVPR 2024 · similarity 0.91)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.90)
- Diffusion Sampling with Momentum for Mitigating Divergence Artifacts (ICLR 2024 · similarity 0.90)
- Accelerating Parallel Sampling of Diffusion Models (ICML 2024 · similarity 0.90)
- Align Your Steps: Optimizing Sampling Schedules in Diffusion Models (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.