Research map / Probabilistic and scientific modeling / Differential equations and PDEs
How to build a consistency model: Learning flow maps via self-distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a self-distillation framework that trains two-time flow maps (consistency-style models) directly from data without a pre-trained teacher, via Eulerian, Lagrangian, and progressive objectives, with Lagrangian self-distillation performing best.
Problem
Flow and diffusion models need expensive ODE solves at inference; flow map (consistency) models enable few-step sampling but current training typically requires a pre-trained teacher or complicated, unstable direct-training pipelines.
Approach
Uses the tangent condition (flow map time derivative at s=t equals the velocity field) and the parameterization X_{s,t}(x)=x+(t-s)v_{s,t}(x) to convert any distillation objective (Eulerian, Lagrangian, progressive) into a self-distillation objective trained jointly with flow matching, with stopgradient and learned loss weighting; Lagrangian variant avoids spatial derivatives.
Key results
- LSD achieves FID 8.10 at 1 step and 3.33 at 8 steps on CIFAR-10, outperforming PSD-M and PSD-U at all step counts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LSD instead of the strongest reported alternative | Flow-map self-distillation · Checkerboard, 1 step | KL: 0.0983 -> 0.0864 | Helped |
| Use LSD instead of the strongest reported alternative | Flow-map self-distillation · Checkerboard, 16 steps | KL: 0.0679 -> 0.0710 | Hurt |
| Use LSD instead of PSD-M | Flow-map self-distillation · CIFAR-10, 1 step | FID: 12.81 -> 8.10 | Helped |
| Use LSD instead of PSD-M | Flow-map self-distillation · CIFAR-10, 16 steps | FID: 4.64 -> 3.57 | Helped |
| Use LSD instead of PSD-U | Flow-map self-distillation · CelebA-64, 1 step | FID: 18.81 -> 12.22 | Helped |
| Use LSD instead of PSD-M | Flow-map self-distillation · CelebA-64, 16 steps | FID: 5.09 -> 1.96 | Helped |
| Use LSD instead of PSD-U | Flow-map self-distillation · AFHQ-64, 1 step | FID: 14.50 -> 11.19 | Helped |
| Use LSD instead of PSD-M | Flow-map self-distillation · AFHQ-64, 16 steps | FID: 11.07 -> 5.61 | Helped |
| Use Eulerian self-distillation, which requires spatial derivatives (ablation) | Flow-map self-distillation · Image generation, including CIFAR-10 | training stability | Hurt |
| Use uniform intermediate-step sampling (PSD-U) instead of midpoint sampling (PSD-M) (ablation) | PSD · CelebA-64 | FID | Mixed |
| Use uniform intermediate-step sampling (PSD-U) instead of midpoint sampling (PSD-M) (ablation) | PSD · AFHQ-64 | FID | Mixed |
| Increase sampling steps | LSD and other evaluated methods · Checkerboard | KL | Helped |
Surprising findings
- Eulerian self-distillation diverged on CIFAR-10 even with stopgradient, due to high parameter gradient norms from spatial derivatives.
Limitations
- Experiments were limited to small-scale benchmarks due to computational expense, so components were not systematically ablated.
- Wasserstein guarantees are not available for progressive self-distillation.
- Only the single-model parameterization was tested; guidance and higher-order parameterizations were left to future work.
Experimental setup
Datasets: CIFAR-10 CelebA-64 AFHQ-64 checkerboard
Benchmarks: CIFAR-10 CelebA-64 AFHQ-64 checkerboard
Metrics: FID KL divergence
Baselines: PSD-M PSD-U ESD
Builds on: stochastic interpolants consistency models progressive distillation EDM2 loss weighting flow map matching
Closest papers by meaning
- Align Your Flow: Scaling Continuous-Time Flow Map Distillation (NeurIPS 2025 · similarity 0.93)
- Improving Consistency Models with Generator-Augmented Flows (ICML 2025 · similarity 0.91)
- Path-Guided Flow Matching for Dataset Distillation (NeurIPS 2026 · similarity 0.90)
- Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment (CVPR 2025 · similarity 0.89)
- Truncated Consistency Models (ICLR 2025 · similarity 0.89)
- Expanding Flow Maps (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.