Research map / Image generation / Fast diffusion sampling
Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models Better
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LCSC, which merges saved training checkpoints of diffusion and consistency models with linear coefficients found by evolutionary search, cutting training cost and improving final models.
Problem
Diffusion and consistency models use only the last converged checkpoint, leaving intermediate checkpoints underused; training is costly and EMA merging coefficients are heuristic and suboptimal.
Approach
Linearly combine saved training checkpoints (allowing negative coefficients, summing to one) with coefficients optimized by gradient-free evolutionary search on FID, rather than using fixed EMA weights.
Key results
- On CIFAR-10 CD, LCSC at 50K iterations reaches FID 3.10 versus 3.66 for the 800K-iteration EMA model, about 14x training speedup.
- On CIFAR-10 CD, a single-step LCSC model at 800K reaches FID 2.44 versus 2.93 for the 2-step released model.
- LCSC reduces DM NFE from 15 to 9 on CIFAR-10 while matching EMA* FID (3.97 vs 3.96).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply LCSC to CD checkpoints after 50K iterations instead of training to 800K | CIFAR-10 CD · CIFAR-10 | FID: 3.66 -> 3.10 | Helped |
| Apply LCSC to full-training CD checkpoints | CIFAR-10 CD · CIFAR-10 | FID: 3.51 -> 2.44 | Helped |
| Apply LCSC with fewer training iterations | ImageNet-64 CD · ImageNet-64 | FID: 7.30 -> 5.51 | Helped |
| Apply LCSC to CT checkpoints | ImageNet-64 CT · ImageNet-64 | FID: 15.6 -> 10.5 | Helped |
| Use LCSC with batch size 128 instead of standard training with batch size 512 | CIFAR-10 CD · CIFAR-10 | FID: 3.66 -> 3.21 | Helped |
| Apply LCSC to fine-tuned released checkpoints | LSUN-Cat CT · LSUN-Cat | FID: 21.0 -> 17.8 | Helped |
| Apply LCSC to final-stage diffusion model checkpoints | CIFAR-10 DM · CIFAR-10 | FID: 3.96 -> 3.18 | Helped |
| Reduce diffusion model sampling from 15 NFE to 9 NFE with LCSC | CIFAR-10 DM · CIFAR-10 | FID: 3.96 -> 3.97 | No clear effect |
| Apply LCSC to LoRA checkpoints | ImageNet-64 CD LoRA · ImageNet-64 | FID: 6.79 -> 4.21 | Helped |
| Apply LCSC to LoRA checkpoints | ImageNet-64 CT LoRA · ImageNet-64 | FID: 15.14 -> 4.90 | Helped |
| Apply LCSC to LCM-LoRA checkpoints | LCM-LoRA · MS-COCO text-to-image | FID: 32.52 -> 28.30 | Helped |
Surprising findings
- Optimal merging coefficients include multiple large negative weights, so the merged model lies outside the convex hull of checkpoints, which EMA cannot reach.
- Linear combinations of checkpoints are reported to reach high-quality basins not reliably found by SGD or Adam.
Limitations
- Search requires saving many intermediate checkpoints and repeated inference for each candidate, which is costly for DM sampling.
- Optimization targets FID on training data, so gains on other metrics are shown only partially.
Experimental setup
Datasets: CIFAR-10 ImageNet-64 LSUN-bedroom LSUN-cat CC12M MS-COCO
Benchmarks: CIFAR-10 ImageNet-64 LSUN-bedroom LSUN-cat MS-COCO
Metrics: FID IS precision recall FCD KID PickScore ImageReward winning rate CLIP-Score
Baselines: EMA EMA* vanilla training LoRA fine-tuning official released CM checkpoints
Builds on: Consistency Models Exponential Moving Average LCM-LoRA DDIM DPM-Solver
Closest papers by meaning
- Simplifying, Stabilizing and Scaling Continuous-time Consistency Models (ICLR 2025 · similarity 0.89)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.87)
- Phased Consistency Models (NeurIPS 2024 · similarity 0.87)
- Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.87)
- Truncated Consistency Models (ICLR 2025 · similarity 0.86)
- Sampling is as easy as keeping the consistency: convergence guarantee for Consistency Models (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.