Research map / Probabilistic and scientific modeling / Variational and generative sampling
Improving Flow Matching by Aligning Flow Divergence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes flow and divergence matching (FDM), adding a conditional divergence loss to conditional flow matching, with a TV-distance bound justifying it, and shows gains on synthetic density, DNA promoter design, dynamical systems and video prediction.
Problem
Conditional flow matching learns vector fields whose divergence can deviate substantially from the true marginal field, leading to inaccurate learned probability paths and poor likelihood estimation.
Approach
Derives a PDE for the error between exact and learned probability paths, bounds the TV gap by a divergence-based loss, and proposes a tractable conditional divergence matching term added to CFM (FDM) with a Hutchinson-trace, stop-gradient efficient variant.
Key results
- On KTH, latent FDM reaches FVD 155.5 vs 180 for latent FM and PSNR 31.2 vs 30.4 (predicting 30 frames from 10).
- On BAIR, latent FDM reaches FVD 123 vs 146 for latent FM, at 35 GB vs 24.2 GB memory.
- On CIFAR-10, FDM (OT) improves FID from 6.35 to 5.62 and NLL from 2.99 to 2.85.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add conditional divergence matching to FM with the OT path | FM · Checkerboard test set | likelihood: 2.38$_{\pm.02}$ -> \textbf{2.53}$_{\pm.02}$ | Helped |
| Add conditional divergence matching to FM with the VP path | FM · Checkerboard test set | likelihood: 2.34$_{\pm.02}$ -> 2.46$_{\pm.02}$ | Helped |
| Add conditional divergence matching to FM | FM (OT) · CIFAR-10 | NLL: 2.99 -> 2.85 | Helped |
| Add conditional divergence matching to FM | FM (OT) · CIFAR-10 | FID: 6.35 -> 5.62 | Helped |
| Add conditional divergence matching to Linear FM | Linear FM · Promoter DNA design with guidance | MSE: 2.82$_{\pm{0.02}}$E-2 -> 2.78$_{\pm{0.01}}$E-2 | Helped |
| Add conditional divergence matching to Dirichlet FM | Dirichlet FM · Promoter DNA design with guidance | MSE: 2.68$_{\pm{0.01}}$E-2 -> \textbf{2.59}$_{\pm{0.02}}$E-2 | Helped |
| Add conditional divergence matching to FM | FM · Lorenz event-conditioned trajectories | TV distance: 0.0972 -> \textbf{0.0914} | Helped |
| Add conditional divergence matching to FM | FM · FitzHugh-Nagumo event-conditioned trajectories | TV distance: 0.2164 -> \textbf{0.1168} | Helped |
| Add conditional divergence matching to FM | FM · Lorenz trajectories | NLL(x1): -13.190 -> \textbf{-14.361} | Helped |
| Add conditional divergence matching to FM | FM · FitzHugh-Nagumo trajectories | NLL(x1): -13.942 -> \textbf{-14.408} | Helped |
| Add conditional divergence matching to Latent FM | Latent FM · KTH video prediction | FVD: 180 -> \textbf{155.5}$_{\pm 5}$ | Helped |
| Add conditional divergence matching to Latent FM | Latent FM · KTH video prediction | PSNR: 30.4 -> \textbf{31.2} | Helped |
Surprising findings
- Minimizing the conditional divergence loss alone does not yield good results because positive and negative terms cancel, so it must be combined with the CFM loss.
Limitations
- Bounded TV distance does not imply a bound on KL divergence, which the paper leaves open.
- Hyperparameters lambda_1 and lambda_2 are chosen by search without a principled rule.
- Latent FDM on BAIR uses more memory (35 GB vs 24.2 GB) and time than latent FM.
Experimental setup
Datasets: checkerboard (synthetic 2D) CIFAR-10 human promoter database (100k sequences, 1024 bp) Lorenz and FitzHugh-Nagumo simulated trajectories KTH human motion BAIR Robot Pushing
Benchmarks: checkerboard likelihood CIFAR-10 simplex dimension promoter design MSE Lorenz and FitzHugh-Nagumo KTH BAIR
Metrics: likelihood NLL FID TV distance KL divergence MSE FVD PSNR time per iteration
Baselines: Flow matching (OT) Flow matching (VP) Diffusion model Dirichlet FM SRVP SLAMP TriVD-GAN-FP Video Transformer LVT RaMViD Bit Diffusion DDSM
Builds on: conditional flow matching Dirichlet flow matching score-based diffusion models Finzi et al. event-guided diffusion sampling latent flow matching
Closest papers by meaning
- Faster Inference of Flow-Based Generative Models via Improved Data-Noise Coupling (ICLR 2025 · similarity 0.91)
- Optimal Flow Matching: Learning Straight Trajectories in Just One Step (NeurIPS 2024 · similarity 0.90)
- Value Gradient Guidance for Flow Matching Alignment (NeurIPS 2025 · similarity 0.89)
- An Error Analysis of Flow Matching for Deep Generative Modeling (ICML 2025 · similarity 0.89)
- Discrete Flow Matching (NeurIPS 2024 · similarity 0.89)
- On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.