Research map / Image generation / Fast diffusion sampling
Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Post-trains pretrained flow matching models (Flux.1 Dev, SD3.5 Large) into 3-8 step samplers via velocity-space self-distillation with a teacher and EMA student, using LoRA and a small LAION-POP subset on one A100.
Problem
Few-step sampling of large pretrained flow matching diffusion models requires either step-size-conditioned retraining (shortcut models), which is nearly as costly as pretraining, or expensive progressive/sample-space distillation that needs large datasets and adversarial losses.
Approach
Distill in velocity space with a dual-target objective: a teacher-based term and a self-distillation term from an EMA student, with step-size implicitly learned and no step-size embedding, so the trajectory is straightened end-to-end.
Key results
- Flux-SCFM at 3 steps reaches FID 6.34 and CLIP 33.10 vs Flux-Schnell 7.06 and 33.06 (FID w.r.t. base in table), trained under 24 A100 GPU hours.
- Flux-SCFM at 8 steps: FID w.r.t. base 2.58 and CLIP 33.76, outperforming Hyper-SD (3.20) and TDD (4.02) at 8 steps.
- SD3.5L-SCFM at 4 steps achieves FID w.r.t. base 6.89 vs SD3.5L-Turbo 6.98.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCFM instead of SD3.5L-Turbo at 8 steps | SD3.5 Large · COCO-30k validation, 8 steps | FID: 8.18 -> 2.65 | Helped |
| Use SCFM instead of Flux-Hyper-SD at 8 steps | Flux.1-Dev · COCO-30k validation, 8 steps | FID: 3.20 -> 2.58 | Helped |
| Use SCFM instead of Flux-Schnell at 4 steps | Flux.1-Dev · COCO-30k validation, 4 steps | FID: 6.76 -> 4.50 | Helped |
| Mix teacher and stop-gradient student solvers in the first loss term (ablation) | SCFM · Flux, 8-step training | CLIP convergence | Helped |
| Periodically restart the EMA student every 500 iterations (ablation) | SCFM · Flux, 8-step training | CLIP convergence: 500 iterations | Mixed |
| Use dual fast-slow EMA (μ=0.99 and μ=0.999) instead of a single EMA (ablation) | SCFM · Flux, 8-step training | iterations to convergence: 1000th iteration | Helped |
| Train on only 10 images rather than the full training set (ablation) | SCFM · COCO-30k validation, 8 steps | FID: 2.58 -> 2.94 | Hurt |
| Train on only 10 images rather than the full training set (ablation) | SCFM · COCO-30k validation, 8 steps | CLIP: 33.76 -> 33.75 | No clear effect |
Surprising findings
- Few-shot distillation with only 10 training images yields results comparable to full-dataset training (Flux 8-step CLIP 33.75 vs 33.76).
- Sample-space distillation losses are typically 10-100x smaller than velocity-space losses, so they require much smaller learning rates and more iterations.
Limitations
- SD3.5 student requires two function evaluations per step since CFG is not distilled, so it is slower than CFG-distilled baselines.
- Adversarial (ADD/LADD) components are not used, so one-step generation is not explored.
- Preservation of teacher diversity is not quantified with a metric.
Experimental setup
Datasets: LAION-POP COCO-30k
Benchmarks: COCO-30k
Metrics: FID delta FID (vs base) CLIP score
Baselines: SD3.5-Turbo Flux-Schnell Flux-Hyper-SD Flux-TDD TeaCache
Builds on: shortcut models progressive distillation rectified flow InstaFlow LoRA
Closest papers by meaning
- Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment (CVPR 2025 · similarity 0.93)
- Inductive Moment Matching (ICML 2025 · similarity 0.91)
- Elucidating Flow Matching ODE Dynamics via Data Geometry and Denoisers (ICML 2025 · similarity 0.90)
- One-Step Diffusion Distillation through Score Implicit Matching (NeurIPS 2024 · similarity 0.90)
- One Step Diffusion via Shortcut Models (ICLR 2025 · similarity 0.90)
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.