Research map / Probabilistic and scientific modeling / Variational and generative sampling
Fisher Flow Matching for Generative Modeling over Discrete Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fisher-Flow, a flow-matching generative model over the probability simplex for discrete data that uses the Fisher-Rao geometry, evaluated on DNA, LM1B text, and molecule generation.
Problem
Generative modeling over discrete data is dominated by autoregressive models, while diffusion and flow-matching alternatives lag behind their continuous-data performance.
Approach
Treats discrete data as categorical distributions on the statistical manifold of the simplex and performs flow matching under the Fisher-Rao geometry (Fisher-information metric), giving a principled continuous flow for discrete variables.
Key results
- On LM1B with 327B tokens, Fisher-Flow reaches test perplexity <=22.42 versus MDLM's <=23.00 (110M parameters).
- On the Promoter DNA dataset, Fisher-Flow achieves MSE 0.029 versus 0.056 for Linear FM and 0.034 for Dirichlet FM.
- On molecule generation, Fisher-Flow gets 88.2% molecules stable, below JODO (98.7%) and EquiFM (93.2%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Fisher-Flow instead of MDLM with 33B training tokens | LM1B generative modeling · LM1B test | PPL: ≤ 27.04 -> ≤ 26.51 | Helped |
| Use Fisher-Flow instead of MDLM with 327B training tokens | LM1B generative modeling · LM1B test | PPL: ≤ 23.00 -> ≤ 22.42 | Helped |
| Use Fisher-Flow instead of Linear FM | DNA generation · Promoter | MSE: 0.056 -> 0.029 ± 0.001 | Helped |
| Use Fisher-Flow instead of Dirichlet FM | DNA generation · Promoter | MSE: 0.034 ± 0.001 -> 0.029 ± 0.001 | Helped |
| Use Fisher-Flow instead of Linear FM | DNA generation · Enhancer Melanoma | PPL (LL): ≤ 1.33 -> ≤ 1.4 ± 0.1 | Hurt |
| Use Fisher-Flow instead of Dirichlet FM | DNA generation · Enhancer Melanoma | PPL (LL): ≤ 2.25 ± 0.01 -> ≤ 1.4 ± 0.1 | Helped |
| Use Fisher-Flow instead of Linear FM | DNA generation · Enhancer FlyBrain | PPL (LL): ≤ 1.35 -> ≤ 1.4 ± 0.66 | Hurt |
| Use Fisher-Flow instead of Dirichlet FM | DNA generation · Enhancer FlyBrain | PPL (LL): ≤ 2.25 ± 0.02 -> ≤ 1.4 ± 0.66 | Helped |
| Use Fisher-Flow instead of JODO | molecule generation · Molecules | Molecules Valid (%): 98.9 -> 95.3 | Hurt |
| Use Fisher-Flow instead of EquiFM | molecule generation · Molecules | Molecules Valid (%): 94.4 -> 95.3 | Helped |
Surprising findings
- On molecule generation, Fisher-Flow underperforms the baselines JODO and EquiFM on molecule stability.
Limitations
- Molecule generation results trail strong baselines (JODO, EquiFM).
- Test perplexities for diffusion/flow methods are upper bounds (<=) and some baselines are taken from concurrent work rather than rerun.
Experimental setup
Datasets: LM1B DNA promoter dataset enhancer Melanoma dataset enhancer FlyBrain dataset molecule generation dataset
Benchmarks: LM1B DNA promoter DNA enhancer Melanoma DNA enhancer FlyBrain molecule generation (atom and molecule stability)
Metrics: test perplexity generated perplexity (Gen-PPL) MSE atoms stable % molecules valid % molecules stable %
Baselines: Linear FM Dirichlet FM MDLM SEDD JODO EquiFM D3PM DiffusionBert Diffusion-LM Transformer (autoregressive)
Builds on: flow matching MDLM Dirichlet flow matching
Closest papers by meaning
- Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design (ICML 2024 · similarity 0.90)
- Categorical Flow Matching on Statistical Manifolds (NeurIPS 2024 · similarity 0.89)
- Dirichlet Flow Matching with Applications to DNA Sequence Design (ICML 2024 · similarity 0.89)
- Discrete Flow Matching (NeurIPS 2024 · similarity 0.88)
- Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective (ICLR 2025 · similarity 0.88)
- Improving Flow Matching by Aligning Flow Divergence (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.