Research map / Image generation / Fast diffusion sampling
Bespoke Solvers for Generative Flow Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Bespoke solvers, a learned, consistent ODE solver family built by time and scale reparametrization of sampling paths of a pretrained flow model, trained with an RMSE upper-bound loss using about 80 parameters.
Problem
Sampling from flow and diffusion models requires many function evaluations (NFE) to solve high-dimensional ODEs/SDEs accurately; distillation is costly to train and dedicated solvers still need many steps.
Approach
Parameterize a family of consistent solvers by applying a base Runge-Kutta solver (RK1/RK2) to time-reparametrized and scale-transformed sampling paths, then learn the few transformation parameters for a specific pretrained model by minimizing a tractable upper bound on the global truncation error (RMSE-Bespoke loss).
Key results
- On CIFAR10 with FM-OT, Bespoke RK2 reaches FID 2.73 at 10 NFE and 2.59 at 20 NFE (GT FID 2.57), beating the best dedicated solver baseline (DEIS, 4.17 at 10 NFE).
- On ImageNet-64 with FM/v-CS, Bespoke RK2 achieves FID 2.20 at 10 NFE and 1.71 at 20 NFE versus GT FID 1.68.
- Training a Bespoke solver takes roughly 0.5-2% of the pretrained model's GPU training time, with about 80 learnable parameters at 10 steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RK2-BES instead of DEIS | FM-OT flow model · CIFAR10, 10 NFE | FID: 4.17 to 2.73 | Helped |
| Use RK2-BES instead of the dedicated solver baselines | CIFAR10 flow models · CIFAR10, low NFE | FID | Helped |
| Use RK2 rather than RK1 at the same function-evaluation budget | CIFAR10 and ImageNet-64 flow models · CIFAR10 and ImageNet-64 | validation RMSE | Helped |
| Add scale transformation to time-only optimization (ablation) | RK2-BES · ImageNet-64 FM-OT | FID | Helped |
| Add scale transformation to time-only optimization (ablation) | RK2-BES · ImageNet-64 FM-OT, fewer than 20 NFE | RMSE | Helped |
| Optimize time transformation rather than scale transformation alone (ablation) | RK2-BES · ImageNet-64 FM-OT | FID | Helped |
| Transfer the RK2-BES solver trained on ImageNet-64 (ablation) | ImageNet-128 FM-OT · ImageNet-128 | FID | Mixed |
| Transfer the RK2-BES solver trained on ImageNet-64 (ablation) | ImageNet-128 FM-OT · ImageNet-128 | RMSE | Helped |
| Transfer the RK2-BES solver trained on ImageNet-64 instead of training a bespoke solver on ImageNet-128 (ablation) | ImageNet-128 FM-OT · ImageNet-128 | FID | Hurt |
Surprising findings
- Bespoke solvers trained on ImageNet-64 transfer partially to ImageNet-128, improving RMSE and PSNR over RK2 and improving FID only at higher NFE.
Limitations
- Consistency of the solver relies on assumed Lipschitz constants, with L_tau treated as a fixed hyperparameter (set to 1).
- Distillation methods still achieve better quality at very low NFE, as the paper acknowledges.
- Training requires ground-truth ODE trajectories computed with an adaptive solver, and the naive implementation re-samples the model at each iteration.
Experimental setup
Datasets: CIFAR10 ImageNet-64 ImageNet-128 AFHQ-256
Benchmarks: CIFAR10 ImageNet-64 ImageNet-128 AFHQ-256
Metrics: FID RMSE PSNR
Baselines: RK1 RK2 RK4 DDIM DPM-2 DPM-Solver DEIS GENIE DDSS EDM progressive distillation
Builds on: Runge-Kutta methods flow matching EDM DPM-Solver DDIM
Closest papers by meaning
- Learning to Solve Generative ODEs Beyond the Linear Span (NeurIPS 2026 · similarity 0.91)
- Distilling ODE Solvers of Diffusion Models into Smaller Steps (CVPR 2024 · similarity 0.91)
- Bespoke Non-Stationary Solvers for Fast Sampling of Diffusion and Flow Models (ICML 2024 · similarity 0.90)
- Improving the Training of Rectified Flows (NeurIPS 2024 · similarity 0.89)
- Simple ReFlow: Improved Techniques for Fast Flow Models (ICLR 2025 · similarity 0.88)
- Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.