Research map / Image generation / Fast diffusion sampling
Phased Consistency Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Phased Consistency Models (PCMs), which split the PF-ODE trajectory into sub-trajectories with per-phase self-consistency, optional CFG-free consistency distillation, and a latent adversarial loss, to improve latent consistency models for text-to-image and text-to-video generation.
Problem
Latent consistency models for high-resolution text-conditioned image generation have inconsistent multi-step sampling, poor controllability (limited CFG range, insensitivity to negative prompts), and low quality in few-step settings.
Approach
Partition the PF-ODE trajectory into several sub-trajectories, enforce self-consistency only within each, use an exact-solution epsilon-based parameterization (DDIM-like form without c_skip/c_out), allowing CFG-free teacher ODE solving, and add a latent-space GAN-style adversarial consistency loss.
Key results
- On SD v1-5 COCO-30K one-step, PCM FID-SD 8.27 vs LCM 53.43 and SD-Turbo 10.62, with consistency score 0.81 vs LCM 0.62.
- On SDXL CC12M-30K 16-step, PCM FID-SD 4.92 vs SDXL-LCM 23.33.
- Video generation: PCM 1-step CLIP score 29.88 vs AnimateLCM 25.41.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use phased consistency models instead of LCM | Stable Diffusion v1-5 · COCO-30K, 1 step | FID-SD: 53.43 -> 8.27 | Helped |
| Use phased consistency models instead of LCM | Stable Diffusion v1-5 · CC12M-30K, 1 step | FID-SD: 42.67 -> 7.91 | Helped |
| Use PCM instead of SDXL-Lightning | Stable Diffusion XL · COCO-30K, 1 step | FID-SD: 9.11 -> 9.32 | Hurt |
| Use PCM instead of SDXL-Turbo | Stable Diffusion XL · CC12M-30K, 2 steps | FID-SD: 6.53 -> 7.02 | Hurt |
| Use PCM instead of SDXL-Lightning | Stable Diffusion XL · CC12M-30K, 8 steps | FID-SD: 5.83 -> 5.19 | Helped |
| Use PCM instead of AnimateLCM | AnimateLCM design · UCF-101 captions, 1 step | CLIP Score: 25.41 -> 29.88 | Helped |
| Add adversarial consistency loss (ablation) | PCM · Stable Diffusion XL, 1 step | HPS: 0.251 -> 0.280 | Helped |
| Add adversarial consistency loss (ablation) | PCM · Stable Diffusion XL, 1 step | AES: 4.994 -> 5.620 | Helped |
| Add adversarial consistency loss (ablation) | PCM · Stable Diffusion XL, 1 step | PickScore: 0.206 -> 0.225 | Helped |
| Add adversarial consistency loss (ablation) | PCM · Stable Diffusion XL, 2–16 steps | HPS: 0.275 -> 0.282; 0.281 -> 0.284; 0.283 -> 0.285; 0.283 -> 0.284 | Helped |
| Replace the latent discriminator with a DINO pixel discriminator (ablation) | PCM adversarial consistency design · Image generation | visual detail | Hurt |
| Replace the proposed adversarial loss with normal GAN loss (ablation) | PCM adversarial consistency design · Image generation | training stability | Hurt |
Surprising findings
- Consistency distillation without CFG in the teacher ODE solver works for PCM with multiple sub-trajectories but not for original CMs.
- Normal GAN loss in consistency distillation causes instability and corrupts generation, while the proposed adversarial loss is needed for good 1-step quality (PCM w/o adv 1-step HPS 0.251 vs 0.280).
Limitations
- One-step generation quality is unstable, producing structural errors or blurry images.
- CFG-free distillation only works with multiple sub-trajectories (M large, M=4 used); not viable for original CMs.
Experimental setup
Datasets: CC3M WebVid-2M
Benchmarks: COCO-2014 30K CC12M-30K UCF-101 captions
Metrics: FID FID-CLIP FID-SD CLIP score CLIP consistency flow magnitude HPSv2 PickScore LAION aesthetic score
Baselines: InstaFlow SD-Turbo LCM CTM TCD SDXL-Turbo SDXL-Lightning SDXL-LCM SDXL-CTM AnimateLCM DDIM DPM
Builds on: consistency models latent consistency models consistency trajectory models AnimateLCM DDIM
Closest papers by meaning
- Simplifying, Stabilizing and Scaling Continuous-time Consistency Models (ICLR 2025 · similarity 0.90)
- Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.90)
- CCM: Real-Time Controllable Visual Content Creation Using Text-to-Image Consistency Models (ICML 2024 · similarity 0.88)
- Theory of Consistency Diffusion Models: Distribution Estimation Meets Fast Sampling (ICML 2024 · similarity 0.88)
- Sampling is as easy as keeping the consistency: convergence guarantee for Consistency Models (ICML 2024 · similarity 0.88)
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.