Research map / Image generation / Fast diffusion sampling

Theory of Consistency Diffusion Models: Distribution Estimation Meets Fast Sampling

ICML 2024 · Image generation › Fast diffusion sampling · approach: Fast diffusion ODE solvers

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Provides the first statistical theory of consistency models, framing their training as Wasserstein distribution-discrepancy minimization and deriving W1 estimation rates for both distillation and isolation training.

Problem

Consistency models accelerate diffusion sampling, but their statistical error rate for estimating the data distribution and how it compares to vanilla diffusion models is unknown.

Approach

Formulates consistency-model training as minimizing a sum of Wasserstein-1 distances between pushed-forward distributions along the probability-flow ODE trajectory, then bounds the statistical error of the learned one-step generator for both distillation (with a pretrained score) and isolation (with the empirical score, no pretraining).

Key results

• Distillation: expected W1 error bound of order sqrt(log n) * n^{-1/(2(d+5))} up to log factors, matching vanilla diffusion model rates.
• Isolation: expected W1 error bound of order n^{-1/d} (up to log factors) without any pretrained score model.
• The distillation error is dominated by score estimation error, so consistency distillation preserves the distribution estimation ability of diffusion models.

Measured effects reported in the paper

Limitations

• Relies on strong assumptions: Lipschitz score, Lipschitz consistency function class, and Gaussian tail or bounded support on the data.
• Isolation analysis requires bounded support, a stronger assumption than the distillation setting.
• Rates have exponential dependence on dimension d, and the bounds are for an idealized global optimum rather than practical optimization.

Experimental setup

Metrics: Wasserstein-1 distance (theoretical bound)

Baselines: vanilla diffusion models

Builds on: consistency models DDPM probability flow ODE Chen et al. 2023 diffusion statistical theory

Closest papers by meaning

• Convergence of Consistency Model with Multistep Sampling under General Data Assumptions (ICML 2025 · similarity 0.91)
• Simplifying, Stabilizing and Scaling Continuous-time Consistency Models (ICLR 2025 · similarity 0.90)
• The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.90)
• Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.89)
• Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.89)
• Truncated Consistency Models (ICLR 2025 · similarity 0.89)