Research map / Image generation / Fast diffusion sampling
Denoising Task Difficulty-based Curriculum for Training Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes denoising task difficulty across diffusion timesteps via convergence speed and relative entropy, then trains diffusion models with an easy-to-hard timestep curriculum using loss-based patience pacing.
Problem
Conflicting claims in the literature about whether low or high timesteps are harder denoising tasks in diffusion model training, and whether training should follow a fixed simultaneous schedule across all timesteps.
Approach
Measure task difficulty via per-timestep-interval convergence speed of loss and FID plus KL divergence between consecutive marginals, conclude earlier (low-t) timesteps are harder, and train with a curriculum that adds timestep clusters from easy (high t) to hard (low t) with loss-plateau-based pacing before standard all-timestep training.
Key results
- On class-conditional ImageNet 256x256 with DiT-B/2, the curriculum reduces FID from 30.27 (vanilla) to 22.96 and raises IS from 60.06 to 75.98.
- With DiT-L/2 at 2M iterations, FID improves from 7.84 (vanilla) to 6.24 with the curriculum.
- Combined with MinSNR, DiT-B/2 FID drops from 21.88 to 19.36; with DTR, from 15.77 to 15.33.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use difficulty-based easy-to-hard curriculum instead of vanilla training | DiT-B · FFHQ 256×256 | FID: 10.49 -> 7.55 | Helped |
| Use difficulty-based easy-to-hard curriculum instead of vanilla training | DiT-L · ImageNet 256×256 | FID: 11.18 -> 8.18 | Helped |
| Use difficulty-based easy-to-hard curriculum instead of vanilla training | DiT-B · MS-COCO 256×256 | FID: 7.62 -> 7.51 | Helped |
| Use difficulty-based curriculum instead of vanilla training | EDM · CIFAR-10 32×32 | FID: 2.67 -> 2.44 | Helped |
| Use difficulty-based curriculum instead of vanilla training | SiT-B, velocity prediction · FFHQ 256×256 | FID: 7.44 -> 6.95 | Helped |
| Replace vanilla training with fixed-iteration NaiveCL (ablation) | DiT-L · ImageNet 256×256 | FID: 11.18 -> 11.90 | Hurt |
| Replace vanilla training with fixed-iteration NaiveCL (ablation) | EDM · FFHQ 64×64 | FID: 2.93 -> 3.13 | Hurt |
| Train clusters hard-to-easy rather than easy-to-hard, with uniform clustering (ablation) | DiT-B/2 · ImageNet 256×256 | FID: 30.27 -> 31.12 | Hurt |
| Use easy-to-hard curriculum rather than vanilla training, with uniform clustering (ablation) | DiT-B/2 · ImageNet 256×256 | FID: 30.27 -> 25.01 | Helped |
| Use SNR-based rather than uniform clustering in the easy-to-hard curriculum (ablation) | DiT-B/2 · ImageNet 256×256 | FID: 25.01 -> 22.96 | Helped |
| Use the curriculum with MinSNR loss weighting | DiT-B/2 · ImageNet 256×256 | FID: 21.88 -> 19.36 | Helped |
| Use the curriculum with DTR | DiT-B/2 · ImageNet 256×256 | FID: 15.77 -> 15.33 | Helped |
Surprising findings
- Naive curriculum (NaiveCL) fails to consistently beat vanilla training; e.g., DiT-L on ImageNet FID worsens from 11.18 to 11.90.
- Anti-curriculum (hard-to-easy) also improves over vanilla in some settings, though less consistently.
Limitations
- Pacing relies on a fixed maximum patience hyperparameter tau and cluster count N that must be tuned.
- Difficulty analysis relies on convergence speed and KL estimates via Monte Carlo that are unstable near t=0.
- Authors note that more dynamic, model-adaptive curricula (self-pacing) are left for future work.
Experimental setup
Datasets: FFHQ CIFAR-10 ImageNet MS-COCO
Benchmarks: FFHQ ImageNet 256x256 CIFAR-10 MS-COCO
Metrics: FID IS Precision Recall
Baselines: Vanilla diffusion training NaiveCL anti-curriculum MinSNR DTR
Builds on: DiT EDM SiT Min-SNR weighting DTR
Closest papers by meaning
- Denoising Task Routing for Diffusion Models (ICLR 2024 · similarity 0.92)
- Adaptive Non-Uniform Timestep Sampling for Accelerating Diffusion Model Training (CVPR 2025 · similarity 0.91)
- Scaling Inference Time Compute for Diffusion Models (CVPR 2025 · similarity 0.91)
- Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (ICLR 2025 · similarity 0.90)
- Decouple-Then-Merge: Finetune Diffusion Models as Multi-Task Learning (CVPR 2025 · similarity 0.90)
- Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.