Research map / Probabilistic and scientific modeling / Variational and generative sampling
Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes kinetic-optimal velocities and probability paths for discrete flow matching over CTMCs, enabling arbitrary corruption processes, and validates on text, crystal, and image generation.
Problem
Discrete flow/diffusion generative models have a limited design space, mostly restricted to masking as the corruption process; general probability paths lack tractable velocities and training objectives.
Approach
Derive closed-form, numerically stable CTMC velocities that generate any given discrete probability path by minimizing a symmetric discrete kinetic energy, plus a kinetic-optimal mixture path with x1-dependent schedulers, and a general ELBO for discrete flow matching.
Key results
- Kinetic-optimal mixture scheduler with beta0=1024 on FineWeb-Edu reaches ELBO 58.41 on LAMBADA and 18.63 on the FineWeb-Edu train set, outperforming the mask-based MD4 and SEDD bounds.
- Permutation-invariant discrete flow matching with kinetic-optimal mask scheduler achieves 7.02% stability rate on MP-20, versus 5.06% for DiffCSP and 4.65% for FlowMM.
- Metric-induced discrete flow matching on ImageNet-256 latent space reaches FID 4.50 at 100 NFE, versus 5.46 for LlamaGen AR and 5.72 for the mask variant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the kinetic-optimal scheduler instead of the linear scheduler | DFM text models · FineWeb-Edu text evaluation | generative perplexity–ELBO tradeoff | Helped |
| Use a non-mask source distribution (β₀=1024) with the kinetic-optimal scheduler instead of a mask source | DFM text models · LAMBADA | perplexity bound: ≤58.5 -> ≤58.41 | Helped |
| Use the kinetic-optimal scheduler instead of the cubic scheduler (ablation) | Permutation-invariant DFM for materials · MP-20 | Stability Rate (%): 6.90 -> 7.02 | Helped |
| Use the metric probability path instead of the masked construction | DFM for face-blurred ImageNet-256 · Face-blurred ImageNet-256 | FID: 5.72 -> 4.50 | Helped |
Surprising findings
- A non-mask source distribution (uniform over least probable tokens) achieves ELBO comparable to or better than the mask source distribution in text.
Limitations
- Text models are small (150M parameters) and trained for a limited budget.
- Kinetic-optimal velocity is derived for symmetric weights; general asymmetric case requires numerical solution of a linear system.
Experimental setup
Datasets: OpenWebText FineWeb-Edu WikiText-103 WikiText-2 LAMBADA Penn Treebank One Billion Words MP-20 CIFAR-10 face-blurred ImageNet
Benchmarks: WikiText-103 WikiText-2 LAMBADA Penn Treebank One Billion Words OpenWebText MP-20 material generation CIFAR-10 FID ImageNet-256 FID
Metrics: ELBO generative perplexity stability rate validity coverage FID NFE
Baselines: SEDD MD4 CDVAE DiffCSP FlowMM CrystalLLM LlamaGen CIFAR-10 discrete baselines
Builds on: discrete diffusion (SEDD) MD4 flow matching discrete flow matching
Closest papers by meaning
- Discrete Flow Matching (NeurIPS 2024 · similarity 0.91)
- Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design (ICML 2024 · similarity 0.90)
- Unlocking Guidance for Discrete State-Space Diffusion and Flow Models (ICLR 2025 · similarity 0.89)
- DeFoG: Discrete Flow Matching for Graph Generation (ICML 2025 · similarity 0.88)
- Flow matching achieves almost minimax optimal convergence (ICLR 2025 · similarity 0.88)
- D-Flow: Differentiating through Flows for Controlled Generation (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.