Research map / Image generation / Fast diffusion sampling
Truncated Consistency Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Truncated Consistency Models (TCM), a two-stage method that trains consistency models only on a truncated time range [t', T] with a boundary loss anchored to a pretrained consistency model, improving one- and two-step FID on CIFAR-10 and ImageNet 64x64.
Problem
Standard consistency models must learn both denoising at small noise levels and generation from pure noise, and this shared capacity limits one-step generation quality and causes training instability.
Approach
Generalize consistency training to a truncated time range [t', T] with a new parameterization that uses a frozen stage-1 consistency model for t < t', plus a boundary loss at t' that prevents collapse to a constant; the stage-1 model supplies the boundary condition.
Key results
- On CIFAR-10, TCM reaches one-step FID 2.46 vs iCT 2.83 and two-step FID 2.05 vs ECM 2.11 with the same 55.7M parameters.
- On ImageNet 64x64 with EDM2-S (280M), TCM reaches one-step FID 2.88 vs iCT-deep 3.25 (592M) and two-step FID 2.31 vs iCT-deep 2.77.
- With EDM2-XL (1119M), TCM reaches one-step FID 2.20 on ImageNet 64x64, versus ECM 2.49.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with truncated consistency training after standard consistency pretraining | Consistency model on CIFAR-10 · CIFAR-10, 1-step | FID: 2.83 -> 2.46 | Helped |
| train with truncated consistency training after standard consistency pretraining | Consistency model on CIFAR-10 · CIFAR-10, 2-step | FID: 2.46 -> 2.05 | Helped |
| train with truncated consistency training after standard consistency pretraining | EDM2-S consistency model · ImageNet 64×64, 1-step | FID: 3.25 -> 2.88 | Helped |
| train with truncated consistency training after standard consistency pretraining | EDM2-S consistency model · ImageNet 64×64, 2-step | FID: 2.77 -> 2.31 | Helped |
| train with truncated consistency training after standard consistency pretraining | Consistency model on MSCOCO · MSCOCO, zero-shot | FID: 18.32 -> 15.58 | Helped |
| set dividing time t' to 1.0 rather than 0.17, 0.8, or 1.5 (ablation) | TCM · CIFAR-10 | FID: 2.70, 2.69, 2.56, 2.79 | Helped |
| add a third truncated training stage with t'=4 (ablation) | TCM · CIFAR-10 | FID: 2.46 -> 2.46 | No clear effect |
| add an intermediate stage that fine-tunes the pretrained model on [0, t') (ablation) | TCM · CIFAR-10 | FID: 2.61 -> >2.7 | Hurt |
| use log-Student-t with σ=0.2, ν=0.01 instead of the other tested distributions (ablation) | TCM · CIFAR-10 | FID | Helped |
| use a very small boundary-loss weight w_b=0.01 (ablation) | TCM · CIFAR-10 | FID | Hurt |
| truncate the training range and use the Stage 2 model instead of standard consistency training Stage 1 (ablation) | Consistency model on CIFAR-10 · CIFAR-10 | FID: 2.77 -> 2.46 | Helped |
Surprising findings
- Standard consistency training shows a trade-off where denoising FID at small t increases while generation FID keeps decreasing, indicating capacity is sacrificed for generation.
- Fine-tuning the stage-1 model on the early range [0, t') before truncated training gave a worse final FID (>2.7).
Limitations
- Adds an extra training stage and a frozen pretrained boundary model, increasing per-iteration time (~18%) and memory (~15%).
- One-step quality still has a considerable gap to diffusion models with many steps.
- Dividing time t' and boundary hyperparameters (w_b, rho) were tuned per setting; t'=1 chosen empirically.
Experimental setup
Datasets: CIFAR-10 ImageNet 64x64 COYO MSCOCO
Benchmarks: CIFAR-10 ImageNet 64x64 MSCOCO
Metrics: FID NFE parameter count
Baselines: iCT iCT-deep ECM MultiStep-CD DMD2 SiD TRACT CTM EDM
Builds on: consistency models iCT ECM EDM EDM2
Closest papers by meaning
- Improving Consistency Models with Generator-Augmented Flows (ICML 2025 · similarity 0.93)
- Simplifying, Stabilizing and Scaling Continuous-time Consistency Models (ICLR 2025 · similarity 0.91)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.91)
- Consistency Diffusion Bridge Models (NeurIPS 2024 · similarity 0.91)
- Consistency Models Made Easy (ICLR 2025 · similarity 0.90)
- Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.