Research map / Image generation / Fast diffusion sampling
On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimality
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives approximation, estimation and total-variation rates for in-context conditional diffusion transformers (DiTs) with classifier-free guidance, including latent variants, and claims minimax optimality for unconditional (latent) DiTs under stronger Hölder assumptions.
Problem
Statistical limits (score approximation, score estimation, distribution estimation) of conditional diffusion transformers and their latent variants are not tightly understood; prior DiT bounds are loose and limited to unconditional models.
Approach
Discretize input domains into fine grids and apply term-by-term Taylor expansion to the conditional score under Hölder smoothness (and lower-bounded density for stronger assumption), with a decomposition that yields piecewise-constant transformer approximations and tighter bounds; extend to linear latent subspace data.
Key results
- Conditional score approximation under stronger Hölder assumption achieves O((log(1/eps))^O(1)/sigma_t^2) error, improving on the generic-assumption bound O((log(1/eps))^{d_x}/sigma_t^4).
- Distribution estimation under stronger Hölder assumption gives TV rate n^{-o(1/d_x)} up to logs, and by setting 2*nu_3=1 the unconditional DiT matches the minimax lower bound n^{-beta/(d_x+2beta)}.
- Latent conditional DiTs achieve better approximation rates than pixel-space conditional DiTs (epsilon^{2/(d_x+d_y)} vs epsilon^{1/(d_x+d_y)} in the stated remark).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the stronger Hölder density assumption and its score decomposition instead of the generic Hölder assumption | Conditional DiT · Score approximation | score approximation error: \tilde{\mathcal{O}}(N^{-\frac{\beta}{d_{x}+d_{y}}}) -> \tilde{\mathcal{O}}(N^{-\frac{2\beta}{d_{x}+d_{y}}}) | Helped |
| Use a latent subspace representation of dimension d_0 instead of the original d_x-dimensional input | Conditional DiT · Score approximation | score approximation error: \tilde{\mathcal{O}}( \epsilon^{1/(d_{x} + d_{y})}/\sigma_{t}^{2} ) -> \tilde{\mathcal{O}}( \epsilon^{2/(d_{x} + d_{y})}/ | Helped |
| Use a latent subspace representation with dimension d_0 instead of the original dimension d_x | Conditional DiT · Score estimation | score estimation error | Helped |
| Use a latent subspace representation with dimension d_0 instead of the original dimension d_x | Conditional DiT · Distribution estimation | distribution estimation error (TV) | Helped |
| Use the stronger Hölder assumption with a density lower bound | Latent conditional DiT · Score approximation | score approximation error | Helped |
Limitations
- Minimax optimality for conditional DiTs is not established.
- Adaptive layer norm and cross-attention DiT architectures are not analyzed.
- Minimax optimality under the Lipschitz score assumption is not shown for latent conditional DiTs.
Experimental setup
Baselines: prior DiT statistical analysis (unconditional) ReLU-network conditional diffusion rates
Builds on: classifier-free guidance diffusion transformer (DiT) local diffused polynomial analysis transformer universal approximation
Closest papers by meaning
- On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs) (NeurIPS 2024 · similarity 0.92)
- FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modification (NeurIPS 2024 · similarity 0.89)
- Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers (CVPR 2025 · similarity 0.88)
- Diffusion Transformers for Imputation: Statistical Efficiency and Uncertainty Quantification (NeurIPS 2025 · similarity 0.88)
- DEFT: Efficient Fine-tuning of Diffusion Models by Learning the Generalised $h$-transform (NeurIPS 2024 · similarity 0.88)
- TFG: Unified Training-Free Guidance for Diffusion Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.