Research map / Image generation / Fast diffusion sampling
See Further When Clear: Curriculum Consistency Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Curriculum Consistency Model (CCM), which uses a PSNR-based knowledge discrepancy metric to adaptively choose the distillation target timestep via multi-step teacher iteration, stabilizing consistency distillation for diffusion and flow matching models.
Problem
Consistency distillation suffers from unstable knowledge discrepancy between student and teacher across timesteps, which makes training inefficient and hurts few-step sampling quality and text-image alignment.
Approach
Measure per-timestep knowledge discrepancy (KDC) via PSNR between student and teacher outputs, and adaptively enlarge the distillation interval by iterating the teacher in small steps until KDC exceeds a threshold, so the curriculum difficulty stays balanced across timesteps.
Key results
- Achieves one-step FID of 1.64 on CIFAR-10 and 2.18 on ImageNet 64x64 (conditional), versus OT-CFM teacher at 5.36 with NFE=100.
- On COCO-2014 30K with SDXL, CCM reaches CLIP score 31.73 and FID 20.47, outperforming PCM (31.63, 21.15).
- Ablation on CIFAR-10 shows dynamic CCM strategy reaches FID 9.32 versus 9.89 for best static strategy and 27.19 for dynamic 0.1t.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply curriculum-consistency distillation | OT-CFM · CIFAR-10 | FID: 5.36 -> 1.64 | Helped |
| apply curriculum-consistency distillation | OT-CFM · ImageNet 64×64 | FID: 5.36 -> 2.18 | Helped |
| apply CCM to SD3 | SD3 · COCO-2014 30K | FID: 86.84 -> 21.49 | Helped |
| apply CCM to SD3 | SD3 · COCO-2014 30K | CLIP Score: 26.18 -> 31.41 | Helped |
| apply CCM to SDXL | SDXL · COCO-2014 30K | FID: 21.15 -> 20.47 | Helped |
| add adversarial loss (ablation) | CCM with SD3 · COCO-2014 30K | FID: 21.83 -> 21.49 | Helped |
| add adversarial loss (ablation) | CCM with SDXL · COCO-2014 30K | FID: 21.08 -> 20.47 | Helped |
| use dynamic curriculum target selection instead of fixed target steps (ablation) | CCM · CIFAR-10 | FID: 9.89 -> 9.32 | Helped |
| use opposite curriculum strategy (ablation) | CCM · CIFAR-10 | FID: 9.32 -> 12.66 | Hurt |
| replace the teacher target with ground truth (ablation) | CCM · CIFAR-10 | FID: 9.32 -> 10.67 | Hurt |
| use a single-step iteration to generate the teacher target (ablation) | CCM · CIFAR-10 | FID: 9.32 -> 46.82 | Hurt |
| increase the number of teacher target-generation steps from 1 to 3 at fixed s=0.03 (ablation) | CCM · CIFAR-10 | FID: 11.38 -> 9.89 | Helped |
Surprising findings
- Directly generating the teacher target in one step (single-step iteration) yields FID 46.82, far worse than multi-step generation (about 9.3), indicating the target quality matters more than the step count.
Limitations
- Flow matching models underperform diffusion models due to teacher and model capacity limits, per the authors.
- Each training iteration is more expensive because multi-step teacher iteration increases per-iteration time.
- Performance on text-to-image depends on LoRA training on CC3M and is evaluated at 4 steps only.
Experimental setup
Datasets: CIFAR-10 ImageNet 64x64 CC3M
Benchmarks: CIFAR-10 ImageNet 64x64 COCO-2014 30K T2I-CompBench
Metrics: FID CLIP score PSNR-based KDC
Baselines: CD iCT ECM CTM LCM PCM Hyper-SD Lightning SiD DMD2 PaGoDA OT-CFM
Builds on: Consistency Models iCT ECM Latent Consistency Models Phased Consistency Model torchcfm
Closest papers by meaning
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.88)
- Your Student is Better Than Expected: Adaptive Teacher-Student Collaboration for Text-Conditional Diffusion Models (CVPR 2024 · similarity 0.88)
- Phased Consistency Models (NeurIPS 2024 · similarity 0.86)
- On the Memorization of Consistency Distillation for Diffusion Models (NeurIPS 2026 · similarity 0.86)
- Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.86)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.