Research map / Image generation / Fast diffusion sampling
Dirichlet Flow Matching with Applications to DNA Sequence Design
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dirichlet flow matching on the simplex with Dirichlet mixture probability paths, plus guidance and one-step distillation, for discrete DNA sequence generation.
Problem
Discrete categorical and sequence generation with flow matching is hard because naive linear flow matching on the simplex has discontinuous, pathological marginal vector fields that degrade with larger vocabularies; also no distillation exists for discrete generative models.
Approach
Define conditional probability paths as Dirichlet distributions with parameters 1 + t·e_i that keep full simplex support, derive a matching conditional vector field, train a denoising classifier with cross-entropy, derive a linear map between score and flow enabling classifier and classifier-free guidance, and distill the deterministic ODE map into a one-step student.
Key results
- On melanoma enhancer generation, Dirichlet FM CFG achieves FBD 1.9 vs 36.0 for the autoregressive language model; on fly brain, 1.0 vs 25.2.
- On promoter design, Dirichlet FM reaches MSE .0269 (100 NFE) vs .0333 for the language model (1024 NFE), and distilled one-step Dirichlet FM reaches .0278.
- Dirichlet FM distilled generates sequences in one step with FBD 6.1 (melanoma) and 15.8 (fly brain), close to the non-distilled model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace linear flow matching with Dirichlet flow matching | DNA promoter sequence design · Promoter DNA sequence design | MSE: .0281 -> .0269 | Helped |
| distill the sampling process into a one-step student (ablation) | Dirichlet FM · Promoter DNA sequence design | MSE: .0269 -> .0278 | Hurt |
| distill the sampling process into a one-step student (ablation) | Dirichlet FM · Promoter DNA sequence design | NFE: 100 -> 1 | Helped |
| replace the language model with Dirichlet flow matching | DNA promoter sequence design · Promoter DNA sequence design | MSE: .0333 -> .0269 | Helped |
| replace the language model with Dirichlet flow matching | Unconditional enhancer generation · Melanoma enhancers | FBD: 36.0 -> 5.3 | Helped |
| replace the language model with Dirichlet flow matching | Unconditional enhancer generation · Fly Brain enhancers | FBD: 25.2 -> 15.2 | Helped |
| add classifier-free guidance toward classes sampled at empirical frequencies (ablation) | Dirichlet FM · Melanoma enhancers | FBD: 5.3 -> 1.9 | Helped |
| add classifier-free guidance toward classes sampled at empirical frequencies (ablation) | Dirichlet FM · Fly Brain enhancers | FBD: 15.2 -> 1.0 | Helped |
| distill the sampling process into a one-step student (ablation) | Dirichlet FM · Melanoma enhancers | FBD: 5.3 -> 6.1 | Hurt |
| distill the sampling process into a one-step student (ablation) | Dirichlet FM · Fly Brain enhancers | FBD: 15.2 -> 15.8 | Hurt |
| use linear flow matching instead of Dirichlet flow matching | Categorical distribution modeling as simplex dimension increases · Simplex dimension toy experiment | KL divergence | Hurt |
| use a Transformer architecture instead of the 20-layer convolutional architecture (ablation) | Dirichlet FM enhancer experiments · DNA enhancer generation | performance | Hurt |
Surprising findings
- Linear FM outperforms the language model on promoter MSE and is the only other method beating it, while Dirichlet FM is best overall.
- Unguided Dirichlet FM is worse than linear FM on fly brain FBD (15.2 vs 15.0), so gains depend on dataset and guidance.
Limitations
- Guidance with gamma > 1 can yield negative probabilities requiring simplex projection.
- Transformer architectures performed worse than the 1D convolutional architecture used.
- Enhancer datasets use ATAC-seq labels, which are necessary but not sufficient for enhancer activity.
Experimental setup
Datasets: promoter sequences from human promoter database (FANTOM5 CAGE) Fly Brain enhancer ATAC-seq dataset Human melanoma enhancer ATAC-seq dataset
Benchmarks: promoter profile-conditioned design MSE (Sei-predicted) Fly Brain enhancer FBD Melanoma enhancer FBD toy categorical KL divergence
Metrics: MSE FBD (Frechet Biological distance) KL divergence NFE classifier target probability
Baselines: Language Model Linear FM DDSM Bit Diffusion D3PM-uniform Random Sequence
Builds on: flow matching DDSM D3PM classifier-free guidance Bit Diffusion
Closest papers by meaning
- Fisher Flow Matching for Generative Modeling over Discrete Data (NeurIPS 2024 · similarity 0.89)
- Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design (ICML 2024 · similarity 0.87)
- Improving Flow Matching by Aligning Flow Divergence (ICML 2025 · similarity 0.85)
- Absorb & Escape: Overcoming Single Model Limitations in Generating Heterogeneous Genomic Sequences (NeurIPS 2024 · similarity 0.85)
- Unlocking Guidance for Discrete State-Space Diffusion and Flow Models (ICLR 2025 · similarity 0.85)
- Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.