Research map / Probabilistic and scientific modeling / Molecules and proteins
Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Discrete Flow Models (DFMs), a CTMC-based flow-matching framework for discrete data, and combines it with FrameFlow into Multiflow for joint protein structure-sequence co-design.
Problem
Flow-based generative models have not been defined for discrete state spaces, blocking a unified flow framework for multimodal (discrete + continuous) data such as protein structure and sequence co-design.
Approach
Define a discrete generative flow via conditional flows that linearly interpolate to data (uniform or mask), sample with a CTMC whose rate matrix is an expectation over a conditional rate matrix, and add a detailed-balance term (eta) that controls CTMC stochasticity at inference; combine with FrameFlow for continuous structure in a single multimodal model.
Key results
- On co-design with ESMFold self-consistency, Multiflow reaches 0.86 designability vs 0.63 for Protpardelle and 0.37 for ProteinGenerator (Co-design 1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| tune CTMC stochasticity at sampling time (best text frontier at η=15) | DFM · text8 | GPT-J-6B NLL versus token entropy | Helped |
| set CTMC stochasticity η=0 | DFM · text8 | GPT-J-6B NLL versus token entropy | No clear effect |
| add distillation (ablation) | Multiflow · Co-design 1 | designability: 0.42 -> 0.86 | Helped |
| add distillation (ablation) | Multiflow · Co-design 1 | diversity: 72 -> 143 | Helped |
| add distillation (ablation) | Multiflow · PMPNN 8 | designability: 0.89 -> 0.99 | Helped |
| add distillation (ablation) | Multiflow · PMPNN 8 | diversity: 126 -> 156 | Helped |
| add sequence generation to the structure-only model (ablation) | Multiflow · PMPNN 8 | diversity: 116 -> 156 | Helped |
| add sequence generation to the structure-only model (ablation) | Multiflow · PMPNN 1 | diversity: 97 -> 143 | Helped |
| use Multiflow co-designed sequences and structures rather than a structure-only model with PMPNN 1 | Multiflow · Co-design 1 versus PMPNN 1 | designability: 0.88 vs 0.88 | Mixed |
| use the co-design model without additional training for forward folding | Multiflow · Forward folding | RMSD: 2.7 $\pm$ 3.9 vs 15.3 $\pm$ 4.5 | Hurt |
Surprising findings
- Distillation with ProteinMPNN-designed sequences and self-generated structures raises co-design designability from 0.42 to 0.86, indicating original PDB sequences were poorer than PMPNN sequences.
- Multiflow's forward folding performance is poor compared with ESMFold (RMSD 15.3 vs 2.7).
Limitations
- Multiflow performs poorly on forward folding compared with ESMFold.
- Side-chain atoms are not modeled; only backbone structure.
- Co-design sequence quality depends on distillation from ProteinMPNN; original PDB sequences underperform.
Experimental setup
Datasets: PDB text8
Benchmarks: text8 RFdiffusion protein benchmark forward/inverse folding test set of 449 proteins
Metrics: NLL under GPT-J-6B entropy designability (scRMSD < 2 Å) diversity (FoldSeek clusters) novelty (TM-score) RMSD scRMSD
Baselines: D3PM SEDD Protpardelle ProteinGenerator RFdiffusion ProteinMPNN ESMFold
Builds on: flow matching D3PM FrameFlow ReST score entropy discrete diffusion
Closest papers by meaning
- Unlocking Guidance for Discrete State-Space Diffusion and Flow Models (ICLR 2025 · similarity 0.91)
- Fisher Flow Matching for Generative Modeling over Discrete Data (NeurIPS 2024 · similarity 0.90)
- Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective (ICLR 2025 · similarity 0.90)
- SE(3)-Stochastic Flow Matching for Protein Backbone Generation (ICLR 2024 · similarity 0.89)
- Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction (ICLR 2025 · similarity 0.89)
- Variable-Length Generative Protein Design via Generalized Poisson Flow (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.