Research map / Image generation / Fast diffusion sampling
PCM : Picard Consistency Model for Fast Parallel Sampling of Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Picard Consistency Model (PCM), trained to predict the fixed-point diffusion output from any Picard iterate, plus model switching (feature or LoRA weight space) to guarantee exact convergence, yielding up to 2.71x speedup over sequential sampling.
Problem
Diffusion sampling is slow due to sequential denoising; Picard-iteration-based parallel sampling (ParaDiGMS) guarantees exact convergence but often converges slowly, so latency gains are limited.
Approach
Train the diffusion model with a consistency-style loss to jump from any Picard iterate directly to the fixed-point solution, then gradually switch back to the original model's outputs (via output-space interpolation or LoRA weight scaling) so the final result converges exactly to the original model's output.
Key results
- On PushT, PCM-LoRA reaches reward 0.83 in 8 steps with 2.71x speedup over sequential DDIM (25 steps, reward 0.80).
- On LDM-CelebA with DDIM, PCM-LoRA at 2 Picard steps achieves FID 67.74 versus 257.83 for Picard, and 6-step PCM-LoRA matches sequential FID (35.97 vs 36.09) at 2.11x speedup.
- On Stable Diffusion v1.4, PCM-LoRA at k=8 reaches CLIP score 26.35 with 3.13x theoretical speedup, exceeding Picard at k=9 (26.34).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train PCM to predict the fixed-point solution from intermediate Picard iterates | Picard sampling on LDM-CelebA · LDM-CelebA, DDIM, same sequential step k=2 | FID: 257.83 -> 77.62 | Helped |
| Use PCM-LoRA instead of PCM | LDM-CelebA · LDM-CelebA, DDIM, 18 sequential steps | speedup: 1.51x -> 2.11x | Helped |
| Use PCM-LoRA instead of PCM | PushT · PushT, DDIM, 25 sequential steps | speedup: 2.55x -> 2.71x | Helped |
| Add variance-based loss weighting (ablation) | PCM · PushT, DDIM, k=8 | Reward: 0.71 -> 0.83 | Helped |
| Replace variance-based loss weighting with 1/k weighting (ablation) | PCM · PushT, DDIM, k=8 | Reward: 0.83 -> 0.78 | Hurt |
| Add model switching between PCM and the original model (ablation) | PCM Picard iteration · CelebA, DDIM | convergence error to ground truth | Helped |
| Use a switching stiffness that is too small (ablation) | PCM model switching · LDM-CelebA, DDIM | convergence error to ground truth | Hurt |
| Use a switching stiffness that is too large (ablation) | PCM model switching · LDM-CelebA, DDIM | convergence speed | Hurt |
| Add EMA to PCM training (ablation) | PCM · CelebA, DDIM | training stability | Helped |
Surprising findings
- Naive PCT (without model switching) converges to a different, non-zero error point than the original model, so model switching is needed to recover exact outputs.
- Without loss weighting by alpha(k), training converges to a suboptimal solution (PushT reward 0.71 vs 0.83 with 1/sqrt(Var(k)) weighting).
Limitations
- PCM-based feature-space switching doubles model storage and inference cost, as both trained and base models are needed.
- Full backpropagation through the diffusion model during training is computationally intensive (mitigated by LoRA).
- Speedups are sometimes marginal (e.g., 1.01x-1.15x for some settings) and on SD only theoretical speedup is reported.
Experimental setup
Datasets: CelebA PushT Stable Diffusion prompts
Benchmarks: CelebA PushT
Metrics: FID CLIP score latency speedup theoretical speedup average reward
Baselines: Sequential DDIM Picard iteration (ParaDiGMS) DPM-Solver Newton's method
Builds on: Consistency Models ParaDiGMS LoRA DDIM Diffusion Policy
Closest papers by meaning
- Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity (NeurIPS 2024 · similarity 0.90)
- Simplifying, Stabilizing and Scaling Continuous-time Consistency Models (ICLR 2025 · similarity 0.89)
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.89)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.88)
- Theory of Consistency Diffusion Models: Distribution Estimation Meets Fast Sampling (ICML 2024 · similarity 0.88)
- Consistency Diffusion Bridge Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.