Research map / Image generation / Fast diffusion sampling
Simple ReFlow: Improved Techniques for Fast Flow Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes seven theory-preserving improvements to ReFlow (loss weighting, exponential time distribution, high-pass-filtered loss, lower dropout, forward and projected coupling pairs, sigmoid discretization with DPM-Solver) that yield state-of-the-art 9-NFE neural ODE FIDs on CIFAR10, AFHQv2, FFHQ and ImageNet-64.
Problem
ReFlow straightens generative ODE trajectories for fast sampling, but in practice it degrades sample quality and heuristic fixes (perceptual or pseudo-Huber losses) can break the marginal-preserving theory needed for exact inversion and likelihood.
Approach
Decomposes ReFlow into training dynamics, learning, and inference components and replaces heuristics with theory-preserving choices: a loss weight 1/stopgrad(loss), an exponential time density 10^t, an invertible linear-map loss (I + lambda*HPF) that keeps the minimizer at the posterior mean, lower dropout, forward and Wasserstein-projected coupling pairs, and a sigmoid time discretization with DP
Key results
- Simple ReFlow reaches FID 2.23 on CIFAR10, 2.30 on AFHQv2, 2.84 on FFHQ and 3.49 on ImageNet-64 with 9 NFEs without guidance.
- With guidance, FID improves to 1.98 (CIFAR10), 1.91 (AFHQv2), 2.67 (FFHQ) and 1.74 (ImageNet-64) at 9 NFEs.
- Ablation: combining improved dynamics and learning (DYN+OP+Forward) cuts CIFAR10 FID from 2.83 baseline to 2.38.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Normalize training loss with inverse per-example loss instead of a constant weight (ablation) | ReFlow · CIFAR10 | FID: 2.83 -> 2.61 | Helped |
| Use time density proportional to 10^t rather than lognormal (ablation) | ReFlow with improved loss weighting · CIFAR10 | FID: 3.28 -> 2.62 | Helped |
| Use the loss I + 10 HPF instead of MSE (ablation) | ReFlow · CIFAR10 | FID: 2.62 -> 2.58 | Helped |
| Increase high-pass-filter loss lambda to 1000 (ablation) | ReFlow · CIFAR10 | FID: 2.58 -> 3.20 | Hurt |
| Reduce dropout to the grid-searched optimal rate (ablation) | ReFlow · FFHQ | FID: 4.28 -> 3.60 | Helped |
| Add forward training pairs (ablation) | ReFlow · FFHQ | FID | No clear effect |
| Add projected pairs to improved dynamics and optimal dropout (ablation) | ReFlow · FFHQ | FID: 3.17 -> 3.13 | Helped |
| Use sigmoid discretization with kappa 20 rather than uniform discretization (ablation) | Simple ReFlow inference · CIFAR10 | FID: 2.36 -> 2.23 | Helped |
| Increase sigmoid discretization kappa from 20 to 30 (ablation) | Simple ReFlow inference · CIFAR10 | FID: 2.23 -> 2.45 | Hurt |
| Add guidance | Simple ReFlow · CIFAR10 | FID: 2.23 -> 1.98 | Helped |
Surprising findings
- Lower dropout than the EDM-initialized value is consistently beneficial for ReFlow training, the opposite of what is used for diffusion denoisers.
- EDM discretization, which works well for diffusion ODEs, performs no better than uniform discretization for ReFlow ODEs due to the t_N < 1 mismatch.
Limitations
- Improvements are evaluated only on small-resolution image datasets (32x32 and 64x64).
- Mini-batch OT baselines show unexplained FID degradation relative to independent coupling, leaving comparisons partly open.
- Forward pairs are limited by the number of training images, and projected coupling requires an adversarially approximated divergence and heuristic lambda schedule.
Experimental setup
Datasets: CIFAR10 AFHQv2 FFHQ ImageNet-64
Benchmarks: CIFAR10 AFHQv2 FFHQ ImageNet-64
Metrics: FID straightness
Baselines: ReFlow Pseudo-Huber ReFlow MinCurv OT-CFM FM-OT MOT-512 EDM DPM-Solver AMED-Solver
Builds on: Rectified Flow EDM Flow Matching DPM-Solver Simple Diffusion
Closest papers by meaning
- Improving the Training of Rectified Flows (NeurIPS 2024 · similarity 0.91)
- Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow (ICLR 2025 · similarity 0.91)
- ProReflow: Progressive Reflow with Decomposed Velocity (CVPR 2025 · similarity 0.91)
- Balanced Conic Rectified Flow (NeurIPS 2025 · similarity 0.90)
- Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment (CVPR 2025 · similarity 0.89)
- Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.