Research map / Image generation / Fast diffusion sampling
Score-Optimal Diffusion Schedules
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives a simulation-free, hyperparameter-free cost for DDM discretisation schedules (Stein/Fisher-divergence based) and optimises schedules via a geodesic/energy argument, applicable to pretrained models and online training.
Problem
Choosing the discretisation schedule for sampling denoising diffusion models is largely manual hyperparameter search; no general principled method exists for selecting it.
Approach
Defines an incremental cost for moving between diffusion-path distributions as the work a Langevin corrector must do (a rescaled Stein divergence between p_t' and the predictor-pushed distribution), derives a local cost, and shows the optimal schedule is a constant-speed geodesic in this metric, computable from score estimates alone.
Key results
- Corrector-optimised schedule achieves CIFAR-10 FID 1.99, matching the best hand-tuned EDM rho=7 schedule (1.96) and beating rho=3 (5.47).
- With only 10 steps on CIFAR-10, the corrector-optimised schedule gives FID 2.46 versus 50.75 for rho=3 and 2.70 for rho=7, showing stability at coarse discretisation.
- On the bimodal Gaussian example, the online-learned schedule improves likelihood and reduces score error during training relative to the linear schedule.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the corrector-optimized schedule instead of the ρ=3 schedule | Pretrained diffusion model · CIFAR-10, T=10 | FID: 50.75 -> 2.46 | Helped |
| Use a score-optimized schedule instead of a hand-tuned schedule | Pretrained diffusion model · FFHQ | FID: 2.46 vs 2.42 | Mixed |
| Replace the corrector-optimized schedule with the predictor-optimized schedule | Pretrained diffusion model · CIFAR-10 | FID: 1.99 -> 1.99 | No clear effect |
| Use a corrector-optimized schedule rather than a ρ=3 schedule at low step counts | Pretrained diffusion model · CIFAR-10, T=20 | FID: 3.92 -> 2.02 | Helped |
| Use a corrector-optimized schedule rather than a ρ=7 schedule | Pretrained diffusion model · CIFAR-10 | FID: 2.46 vs 2.70; 2.02 vs 2.00; 2.04 vs 2.06; 2.06 vs 2.05; 2.07 vs 2.07 | Mixed |
Surprising findings
- The KLUB objective from prior work correlates poorly with FID, whereas the proposed cost correlates closely with FID.
- Predictor-optimised and corrector-optimised schedules give similar FID, so the cheaper corrector-optimised cost is recommended for images.
Limitations
- Theory assumes perfect score estimation and that the corrector converges to stationarity (Assumption on corrector).
- Predictor-optimised cost requires second derivatives and is expensive.
- Experiments on image data are limited to pretrained EDM models and small online-training runs on CIFAR-10 and MNIST.
Experimental setup
Datasets: CIFAR-10 MNIST FFHQ AFHQv2 ImageNet 64x64 mollified Cantor distribution bimodal Gaussian
Benchmarks: CIFAR-10 FFHQ AFHQv2 ImageNet 64x64 MNIST
Metrics: FID sFID incremental cost log-likelihood
Baselines: EDM rho=3 schedule EDM rho=7 schedule LogLinear schedule Convex schedule KLUB schedule objective linear schedule cosine schedule
Builds on: EDM score-based SDE annealed Langevin dynamics probability flow ODE Kullback-Leibler upper bound schedule optimisation
Closest papers by meaning
- Align Your Steps: Optimizing Sampling Schedules in Diffusion Models (ICML 2024 · similarity 0.92)
- Multi-Step Denoising Scheduled Sampling: Towards Alleviating Exposure Bias for Diffusion Models (AAAI 2024 · similarity 0.91)
- Learning to Discretize Denoising Diffusion ODEs (ICLR 2025 · similarity 0.91)
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.91)
- Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.90)
- Interpreting and Improving Diffusion Models from an Optimization Perspective (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.