Research map / Probabilistic and scientific modeling / Molecules and proteins
Structure Language Models for Protein Conformation Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Structure Language Modeling (SLM), which generates protein conformations by language modeling over discrete latent structure tokens from a dVAE, instantiated as ESMDiff, a masked-diffusion fine-tune of ESM3.
Problem
Generating diverse protein conformational ensembles efficiently; existing diffusion/flow methods in 3D space are slow and struggle to capture alternative conformations.
Approach
Encode protein structures into discrete latent structure tokens with a dVAE, then model the sequence-conditioned distribution over these tokens with language models (encoder-decoder, decoder-only, or masked-diffusion BERT such as ESMDiff), decoding back to 3D coordinates; this avoids equivariant geometric modeling and is much faster.
Key results
- ESMDiff (DDPM) achieves the best JS-PwD (0.372), JS-TIC (0.420) on BPTI among all methods, versus 0.503 and 0.462 for AlphaFlow.
- SLMs are reported as 20-100x faster than diffusion baselines such as AlphaFlow in runtime profiling for N=100 ensembles.
- On fold-switch pairs, ESMDiff (DDPM) reaches residue-flexibility correlation 0.402 (global), above the MSA-based AlphaFlow's 0.385, while MSA-based methods remain stronger overall on the apo/holo set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune ESM3 with masked diffusion and use DDPM sampling instead of zero-shot ESM3 | ESMDiff on BPTI · BPTI | JS-PwD: 0.406 -> 0.372 | Helped |
| use DDPM sampling rather than iterative decoding | ESMDiff on BPTI · BPTI | JS-PwD: 0.422 -> 0.372 | Helped |
| use DDPM sampling rather than iterative decoding | ESMDiff on BPTI · BPTI | Validity: 0.910 -> 0.940 | Helped |
| use entropy ranking instead of uniform decoding (ablation) | ESM3 zero-shot generation · BPTI | JS-TIC: 0.428 -> 0.402 | Helped |
| use entropy ranking instead of uniform decoding (ablation) | ESM3 zero-shot generation · BPTI | Validity: 0.840 -> 0.740 | Hurt |
| use secondary-structure ranking instead of uniform decoding (ablation) | ESM3 zero-shot generation · BPTI | Validity: 0.840 -> 0.930 | Helped |
| use maxlogit ranking instead of uniform decoding (ablation) | ESM3 zero-shot generation · Apo/holo pairs | ResFlex r (global): 0.223 -> 0.386 | Helped |
| use entropy ranking instead of uniform decoding (ablation) | ESM3 zero-shot generation · Fold-switch pairs | ResFlex r (global): 0.328 -> 0.407 | Helped |
| use entropy ranking instead of uniform decoding (ablation) | ESM3 zero-shot generation · IDP test set | Contact map MAE: 0.328 / 0.264 -> 0.249 / 0.170 | Helped |
| use secondary-structure ranking instead of uniform decoding (ablation) | ESM3 zero-shot generation · IDP test set | Pairwise distance MAE: 6.661 / 4.810 -> 6.365 / 4.767 | Mixed |
| use MSA-based sampling rather than ESMDiff | Apo/holo pair generation · Apo/holo pairs | ResFlex r (global): 0.424 -> 0.455 | Helped |
| use ESMDiff rather than AlphaFlow | Fold-switch pair generation · Fold-switch pairs | ResFlex r (global): 0.385 -> 0.402 | Helped |
Surprising findings
- MSA-based methods generally outperform the SLM family on conformation-changing pairs, which the authors attribute to the value of MSA information.
Limitations
- Relies on a frozen dVAE structure tokenizer; reconstruction fidelity and disentanglement are not optimized.
- Models only backbone atoms; side chains, ligands and RNA not covered.
- MSA-based methods outperform SLMs on conformation-change pairs, suggesting need for MSA-conditioned SLMs.
Experimental setup
Datasets: PDB monomeric structures (up to May 2020) BPTI MD simulation data Protein Ensemble Database (PED) IDPs fold-switching and apo/holo pairs
Benchmarks: BPTI MD ensemble fold-switching and apo/holo conformational pairs PED intrinsically disordered proteins
Metrics: JS divergence (PwD, TIC, Rg) validity ensemble TM-score ensemble RMSD residue flexibility correlation MAE of pairwise distance, Rg, contact map
Baselines: MSA subsampling AlphaFlow EigenFold Str2Str ESMFlow idpGAN
Builds on: ESM3 masked diffusion dVAE T5 GPT-2
Closest papers by meaning
- Diffusion on Language Model Encodings for Protein Sequence Generation (ICML 2025 · similarity 0.91)
- DPLM-2: A Multimodal Diffusion Protein Language Model (ICLR 2025 · similarity 0.89)
- Diffusion Language Models Are Versatile Protein Learners (ICML 2024 · similarity 0.89)
- Protein Conformation Generation via Force-Guided SE(3) Diffusion Models (ICML 2024 · similarity 0.89)
- AlphaFold Meets Flow Matching for Generating Protein Ensembles (ICML 2024 · similarity 0.88)
- Swallowing the Bitter Pill: Simplified Scalable Conformer Generation (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.