Research map / Image generation / Fast diffusion sampling
Improving the Training of Rectified Flows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows a single Reflow round suffices for near-straight rectified flow trajectories and proposes U-shaped timestep sampling and LPIPS-Huber loss, initialized from EDM, so 2-rectified flow++ rivals distillation and iCT in 1-2 NFE image generation.
Problem
Rectified flows need many NFEs for good samples and prior work believed multiple Reflow rounds were needed, leaving them behind distillation methods in the low-NFE regime.
Approach
Argues the optimal 2-rectified flow is nearly straight under realistic data, so one Reflow round suffices; improves its training with a U-shaped timestep distribution, an LPIPS-Huber premetric, and EDM initialization via a converted-time proposition.
Key results
- On CIFAR-10, 2-rectified flow++ reaches 1-NFE FID 3.07 versus 12.21 for the previous 2-rectified flow, a 75% reduction.
- On ImageNet 64x64, 1-NFE FID 4.31 outperforms CD (6.20), PD (15.39) and DFNO (7.83) and is close to iCT (4.02).
- Ablations show the U-shaped timestep distribution and LPIPS-Huber loss each give sizeable gains, e.g. CIFAR-10 1-NFE FID 7.14 to 5.17 and 5.24 to 3.42.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add EDM initialization and increase batch size (ablation) | 2-rectified flow · CIFAR-10 | FID: 12.21 to 7.14 | Helped |
| Add U-shaped timestep distribution (ablation) | Config B 2-rectified flow · CIFAR-10, AFHQ, and FFHQ, 1 NFE | FID: 7.14 to 5.17; 12.39 to 9.03; 8.84 to 6.81 | Helped |
| Replace squared ℓ2 loss with Huber loss (ablation) | Config C 2-rectified flow · CIFAR-10, AFHQ, and FFHQ | FID: 5.17 to 5.24; 9.03 to 8.20; 6.81 to 7.06 | Mixed |
| Add LPIPS-Huber loss instead of Huber (ablation) | 2-rectified flow · CIFAR-10, AFHQ, and FFHQ, 1 NFE | FID: 5.24 to 3.38; 8.20 to 4.11; 7.06 to 5.21 | Helped |
| Weight LPIPS by 1/t in LPIPS-Huber loss (ablation) | Config E 2-rectified flow · CIFAR-10, AFHQ, and FFHQ | FID: 3.42 to 3.38; 4.13 to 4.11; 5.21 to 5.65 | Mixed |
| Fine-tune with real-data and synthetic-noise pairs (ablation) | Config F 2-rectified flow · CIFAR-10, 1 NFE | FID: 3.38 to 3.07 | Helped |
| Use no synthetic-data and real-noise pairs during fine-tuning (ablation) | Config F 2-rectified flow · CIFAR-10 | FID | Helped |
| Use 2-rectified flow++ instead of previous 2-rectified flow | CIFAR-10 · CIFAR-10, 1 NFE | FID: 12.21 -> 3.07 | Helped |
| Use the new history-dependent sampler instead of the standard update rule (ablation) | 2-rectified flow++ · Few-step sampling | sampling efficiency: up to $4\times$ | Helped |
| Use LPIPS loss during training | 2-rectified flow++ · ImageNet | training time per iteration: about 15% | Hurt |
| Use 2-rectified flow++ rather than iCT | ImageNet 64×64 · ImageNet 64×64, 1 NFE | FID: 4.01 vs 4.31 | Hurt |
Surprising findings
- A single Reflow round is sufficient, contrary to prior empirical practice of applying Reflow at least twice.
- Using real data for fine-tuning on CIFAR-10 further reduced 1-NFE FID from 3.38 to 3.07.
Limitations
- Still does not outperform the best consistency model iCT on ImageNet and CIFAR-10 2-NFE.
- LPIPS loss slows training by about 15% per iteration on ImageNet.
- Experiments limited to 64x64 image datasets and single FID runs without repeats.
Experimental setup
Datasets: CIFAR-10 AFHQ FFHQ ImageNet 64x64
Benchmarks: CIFAR-10 ImageNet 64x64 AFHQ 64x64 FFHQ 64x64
Metrics: FID IS precision recall MSE reconstruction error
Baselines: consistency distillation progressive distillation iCT iCT-deep CTM DMD TRACT DFNO 2-rectified flow 3-rectified flow StyleGAN2-ADA BigGAN-deep
Builds on: Rectified flow Reflow EDM LPIPS Pseudo-Huber loss
Closest papers by meaning
- Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow (ICLR 2025 · similarity 0.94)
- Balanced Conic Rectified Flow (NeurIPS 2025 · similarity 0.92)
- Simple ReFlow: Improved Techniques for Fast Flow Models (ICLR 2025 · similarity 0.91)
- ProReflow: Progressive Reflow with Decomposed Velocity (CVPR 2025 · similarity 0.91)
- Text-to-Image Rectified Flow as Plug-and-Play Priors (ICLR 2025 · similarity 0.90)
- SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consistent Trajectories (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.