Research map / Probabilistic and scientific modeling / Variational and generative sampling
Improving Consistency Models with Generator-Augmented Flows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves a continuous-time discrepancy between consistency training and distillation losses and proposes generator-augmented couplings, where a jointly learned consistency model maps IC noisy points to endpoints that are re-coupled with the same noise, improving iCT/ECT on image benchmarks.
Problem
Consistency training (CT) replaces the true velocity field with a one-sample Monte Carlo estimate, so it deviates from consistency distillation (CD) even in continuous time; reducing this discrepancy and data-noise transport cost is the goal.
Approach
Use the consistency model itself as an endpoint predictor on independent-coupling noisy points, then re-couple the predicted endpoint with the same noise vector to build generator-augmented trajectories (GC) for training; this reduces the velocity estimation error and transport cost. Training jointly mixes GC and IC pairs with probability mu.
Key results
- On CIFAR-10, iCT-GC (mu=0.5) reaches FID 5.95 versus 7.42 for iCT-IC and 6.75 for iCT-OT.
- On CelebA 64x64, iCT-GC reaches FID 11.74 versus 15.82 for iCT-IC and 13.63 for iCT-OT.
- In the ECT setting on CIFAR-10 with short training (4k iterations), ECT-GC (mu=0.3) reaches FID 6.41 versus 7.37 for ECT-IC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use joint learning with generator-augmented coupling (μ=0.5) instead of independent coupling | iCT consistency model · CIFAR-10 | FID: 7.42 -> 5.95 | Helped |
| use joint learning with generator-augmented coupling (μ=0.5) instead of minibatch optimal transport | iCT consistency model · CIFAR-10 | FID: 6.75 -> 5.95 | Helped |
| use joint learning with generator-augmented coupling (μ=0.5) instead of independent coupling | iCT consistency model · ImageNet (32×32) | FID: 14.89 -> 13.99 | Helped |
| use joint learning with generator-augmented coupling (μ=0.5) instead of minibatch optimal transport | iCT consistency model · ImageNet (32×32) | FID: 14.13 -> 13.99 | Helped |
| use joint learning with generator-augmented coupling (μ=0.5) instead of independent coupling | iCT consistency model · CelebA (64×64) | FID: 15.82 -> 11.74 | Helped |
| use joint learning with generator-augmented coupling (μ=0.5) instead of minibatch optimal transport | iCT consistency model · CelebA (64×64) | FID: 13.63 -> 11.74 | Helped |
| use joint learning with generator-augmented coupling (μ=0.5) instead of independent coupling | iCT consistency model · LSUN Church (64×64) | FID: 10.58 -> 9.88 | Helped |
| use joint learning with generator-augmented coupling (μ=0.5) instead of minibatch optimal transport | iCT consistency model · LSUN Church (64×64) | FID: 9.71 -> 9.88 | Hurt |
| use generator-augmented coupling (μ=0.3) instead of independent coupling | ECT consistency model · CIFAR-10 (Short Training) | FID: 7.37 -> 6.41 | Helped |
| use generator-augmented coupling (μ=0.3) instead of independent coupling | ECT consistency model · CIFAR-10 (Long Training) | FID: 4.11 -> 3.74 | Helped |
| use generator-augmented coupling (μ=0.3) instead of independent coupling | ECT consistency model · FFHQ 64×64 (Short Training) | FID: 13.29 -> 11.73 | Helped |
| use generator-augmented coupling (μ=0.3) instead of independent coupling | ECT consistency model · FFHQ 64×64 (Long Training) | FID: 9.68 -> 8.51 | Helped |
Surprising findings
- On LSUN Church and ImageNet 64x64 long training, the IC-only ECT baseline matches or beats the GC variant, so gains are not uniform.
- Using mu=1 (only GC pairs) is expected not to work and FID degrades late in training, showing IC pairs are needed.
Limitations
- Theory assumes an ideal consistency model as the endpoint predictor; in practice the jointly learned model is imperfect.
- The discrepancy is only reduced, not eliminated; GC does not fully resolve the alignment issue.
- Gains are mixed across datasets and settings (e.g., LSUN Church iCT-GC does not beat iCT-OT in FID).
Experimental setup
Datasets: CIFAR-10 ImageNet CelebA LSUN Church FFHQ
Benchmarks: CIFAR-10 ImageNet 32x32 CelebA 64x64 LSUN Church 64x64 FFHQ 64x64
Metrics: FID KID IS
Baselines: iCT-IC iCT-OT ECT-IC
Builds on: consistency models improved consistency training (iCT) easy consistency tuning (ECT) minibatch optimal transport coupling rectified flow
Closest papers by meaning
- Truncated Consistency Models (ICLR 2025 · similarity 0.93)
- Consistency Models Made Easy (ICLR 2025 · similarity 0.91)
- How to build a consistency model: Learning flow maps via self-distillation (NeurIPS 2025 · similarity 0.91)
- Elucidating the Preconditioning in Consistency Distillation (ICLR 2025 · similarity 0.91)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.91)
- Convergence of Consistency Model with Multistep Sampling under General Data Assumptions (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.