Research map / Image generation / Fast diffusion sampling
D-Flow: Differentiating through Flows for Controlled Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes D-Flow, which controls a frozen flow/diffusion model by optimizing the source noise point x0 through the ODE sampler to minimize a differentiable cost, and shows this implicitly projects gradients onto the data manifold.
Problem
Controlled generation and inverse problems with pre-trained diffusion/flow-matching models without task-specific retraining or restrictive guidance assumptions.
Approach
Optimize the source noise x0 by differentiating an arbitrary cost through the ODE sampling process of a frozen flow model; for Gaussian-path models, this differentiation implicitly projects the cost gradient onto high-variance data directions, keeping samples near the data manifold without an explicit prior term.
Key results
- On face-blurred ImageNet-128 center inpainting (noiseless), D-Flow reports FID 4.14 vs 5.40 for RED-Diff and 5.73 for PiGDM.
- On QM9 conditional generation, D-Flow reports MAE of 1.39 Bohr^2 for polarizability vs 2.37 for the best baseline (G), and 0.300 D for dipole moment vs 1.106.
- Audio inpainting and super-resolution on MusicCaps attain the lowest FAD in all settings (e.g., 1.19 vs 3.59 for RED-Diff at 10% inpainting).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize the source noise through the flow to perform center inpainting | Cond-OT flow-matching model · ImageNet-128 inpainting-center, σ_y=0 | FID: 5.40 -> 4.14 | Helped |
| Optimize the source noise through the flow for super-resolution | Cond-OT flow-matching model · ImageNet-128 super-resolution ×2, σ_y=0 | FID: 3.05 -> 2.50 | Helped |
| Optimize the source noise through the flow for noisy center inpainting | Cond-OT flow-matching model · ImageNet-128 inpainting-center, σ_y=0.05 | FID: 6.25 -> 4.76 | Helped |
| Optimize the source noise through the flow for noisy super-resolution | Cond-OT flow-matching model · ImageNet-128 super-resolution ×2, σ_y=0.05 | FID: 4.38 -> 4.26 | Helped |
| Optimize the source noise through the flow for free-form inpainting | T2I latent flow-matching model · MS-COCO free-form inpainting | FID: 23.31 -> 16.92 | Helped |
| Optimize the source noise through the flow for music inpainting | Latent music flow-matching model · In-domain music inpainting (10%) | FAD: 0.75 -> 0.22 | Helped |
| Optimize the source noise through the flow for music super-resolution | Latent music flow-matching model · MusicCaps super-resolution ×2 | FAD: 3.07 -> 1.25 | Helped |
| Optimize source noise to condition molecule generation | Unconditional equivariant flow-matching model · QM9 conditional generation, polarizability α | MAE: 2.37 -> {1.39} | Helped |
| Add D-Flow optimization to midpoint sampling (ablation) | Unconditional equivariant flow-matching model · QM9 generation | molecule stability (%): 72.10 -> 58.97 | Hurt |
| Add D-Flow optimization to midpoint sampling (ablation) | Unconditional equivariant flow-matching model · QM9 generation | validity (%): 85.56 -> 79.38 | Hurt |
Surprising findings
- Molecule stability drops from about 72% (unoptimized sampling) to about 59% after D-Flow optimization, a cost of the controlled generation.
Limitations
- Long runtime per sample due to backpropagation through multiple ODE solver steps (5-30+ minutes).
- Stability of generated molecules drops after optimization, and the authors attribute this partly to the unconditional flow model being weaker than EDM.
- Regularization is needed in noisy linear inverse problems and molecule generation.
Experimental setup
Datasets: face-blurred ImageNet-128 MS-COCO validation set MusicCaps QM9
Benchmarks: face-blurred ImageNet-128 (10k split) MS-COCO val (1k subset) MusicCaps QM9
Metrics: FID LPIPS PSNR SSIM Clip score FAD MAE atom stability molecule stability validity
Baselines: PiGDM OT-ODE RED-Diff EDM Equivariant Flow-Matching (E) GeoLDM (G)
Builds on: flow matching RED-Diff torchdiffeq EnCodec
Closest papers by meaning
- Inverse Flow and Consistency Models (ICML 2025 · similarity 0.89)
- Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment (CVPR 2025 · similarity 0.89)
- Drift Flow Matching (NeurIPS 2026 · similarity 0.89)
- Training Free Guided Flow-Matching with Optimal Control (ICLR 2025 · similarity 0.88)
- TFG-Flow: Training-free Guidance in Multimodal Generative Flow (ICLR 2025 · similarity 0.88)
- Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.