Research map / Image generation / Fast diffusion sampling
Accelerating Parallel Sampling of Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ParaTAA, a training-free parallel sampler that reformulates diffusion sampling as triangular nonlinear equations solved by fixed-point iteration with a triangular Anderson acceleration, cutting sequential steps 4-14x.
Problem
Diffusion sampling is autoregressive and slow; existing parallel sampling (Picard iteration) needs many iterations to converge, so the number of sequential network evaluation rounds remains high.
Approach
Reformulate sequential DDIM/DDPM sampling as a system of triangular nonlinear equations of order k solved by fixed-point iteration, then accelerate it with a Triangular Anderson Acceleration whose update matrix is constrained to be block upper triangular, plus early stopping and trajectory initialization.
Key results
- ParaTAA reduces sequential steps by 4-14x across DiT and SD with DDIM-25/50/100 and DDPM-100 (e.g., DiT DDIM-100 from 100 to 11 steps).
- With SD DDIM-100, ParaTAA produces images matching sequential sampling in only 7 inference steps.
- Wall-clock speedup of 1.5-2.9x on 8 A800 GPUs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ParaTAA instead of sequential sampling | DiT and Stable Diffusion v1.5 · DDIM/DDPM sampling scenarios | inference steps: 4$\sim$14x | Helped |
| Use ParaTAA instead of FP+ | DiT DDIM-25 · DiT DDIM-25 | inference steps: 13 -> 9 | Helped |
| Use ParaTAA instead of FP+ | DiT DDIM-50 · DiT DDIM-50 | inference steps: 17 -> 9 | Helped |
| Use ParaTAA instead of FP+ | DiT DDIM-100 · DiT DDIM-100 | inference steps: 19 -> 11 | Helped |
| Use ParaTAA instead of FP+ | DiT DDPM-100 · DiT DDPM-100 | inference steps: 31 -> 21 | Helped |
| Use ParaTAA instead of FP+ | Stable Diffusion v1.5 · DDIM-25, DDIM-50, and DDIM-100 | inference steps: 7 -> 7 | No clear effect |
| Use ParaTAA instead of FP+ | Stable Diffusion v1.5 · DDPM-100 | inference steps: 21 -> 15 | Helped |
| Add Triangular Anderson Acceleration to fixed-point iteration (ablation) | FP for DiT DDIM/DDPM-100 · DiT DDIM-100 and DDPM-100 | convergence speed | Helped |
| Use triangular rather than standard Anderson Acceleration (ablation) | Anderson acceleration for triangular equations · DiT DDIM-100 and DDPM-100 | convergence speed | Helped |
| Use the optimal equation order k rather than k equal to the window size (ablation) | Fixed-point iteration · DDIM/DDPM sampling | inference steps | Helped |
| Increase window size from w=10 to w=20 (ablation) | ParaTAA with Stable Diffusion · SD DDIM-100 | inference steps: 25 -> 21 | Helped |
| Initialize from the similar prompt's existing sampling trajectory rather than random noise (ablation) | ParaTAA for prompt P2 · Stable Diffusion DDIM-50 prompt transfer | CLIP Score convergence | Helped |
Surprising findings
- DDPM (stochastic) needs more parallel steps to converge than DDIM, suggested to stem from the noise term worsening the optimization landscape.
- Plain Anderson Acceleration was numerically unstable in 16-bit precision, while the triangular variant remained stable.
Limitations
- Wall-clock speedup requires multiple GPUs; large models like SD need parallel hardware for considerable gains.
- Only first-order solvers (DDIM, DDPM) studied; higher-order solvers left as future work.
- No theoretical convergence analysis of triangular Anderson acceleration beyond the T-step safeguard.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet 256x256 (FID, IS) with DiT Stable Diffusion CLIP Score on random color-animal prompts
Metrics: FID Inception Score CLIP Score inference steps wall-clock time
Baselines: Sequential sampling FP (fixed-point iteration, ParaDiGMS-style) FP+
Builds on: Picard-Lindelöf iteration parallel sampling (ParaDiGMS) Anderson Acceleration DDIM DDPM SDEdit
Closest papers by meaning
- Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity (NeurIPS 2024 · similarity 0.92)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.91)
- TADA: Improved Diffusion Sampling with Training-free Augmented DynAmics (NeurIPS 2025 · similarity 0.90)
- Simple and Fast Distillation of Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models (ICML 2025 · similarity 0.89)
- Fast constrained sampling in pre-trained diffusion models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.