Research map / Probabilistic and scientific modeling / Molecules and proteins
Protein Design with Dynamic Protein Vocabulary
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ProDVa, a GPT-style protein language model augmented with a dynamic protein vocabulary of InterPro-derived fragments retrieved by text similarity, to design well-folded proteins from function text.
Problem
Function-based protein design from textual descriptions often yields sequences that fail to fold (low pLDDT, high PAE) despite matching the requested function.
Approach
Augments a protein language model's fixed amino-acid vocabulary with dynamically retrieved functional protein fragments (identified by InterPro) whose embeddings are added to the input and output vocabularies, trained with type and description contrastive losses; fragments are retrieved at inference via text-embedding similarity to the input description.
Key results
- On the CAMEO subset, ProDVa reaches pLDDT 75.88 and 77.00% with pLDDT>70 versus Pinal's 66.50 and 47.21%, using 392K pairs versus Pinal's 1.76B.
- On Mol-Instructions, ProDVa achieves pLDDT 76.86 with 76.35% above 70 and PAE 8.66, the best foldability among compared models, while Pinal has ProTrek Score 17.50 vs ProDVa 17.40.
- In unconditional generation, ProDVa attains pLDDT 77.52 and 79.78% above 70, versus Pinal's 70.11 and 57.02%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Incorporate randomly selected natural-protein fragments into random amino-acid sequences | Random (E) generation · CAMEO subset | pLDDT: 27.46 -> 62.38 | Helped |
| Incorporate randomly selected natural-protein fragments into random amino-acid sequences | Random (E) generation · CAMEO subset | PAE: 23.70 -> 17.23 | Helped |
| Use ProDVa instead of Pinal | Function-keyword protein design · CAMEO subset | proteins with pLDDT > 70 (%): +29.79% | Helped |
| Use ProDVa instead of Pinal | Function-keyword protein design · CAMEO subset | ProTrek Score: 14.57% and 14.43%; increase of 0.14% for Pinal | Hurt |
| Use ProDVa instead of Pinal | Textual-description protein design · Mol-Instructions | proteins with pLDDT > 70 (%): 7.38% | Helped |
| Use ProDVa instead of Pinal | Textual-description protein design · Mol-Instructions | ProTrek Score: 17.40 vs 17.50; gap of 0.1% | Hurt |
| Add fragment vocabulary and functional-annotation learning to the vanilla multimodal model (ablation) | ProDVa · Mol-Instructions | proteins with pLDDT > 70 (%): +13.66% | Helped |
| Add fragment vocabulary and functional-annotation learning to the vanilla multimodal model (ablation) | ProDVa · Mol-Instructions | ProTrek Score: 7.77% | Helped |
| Include both type loss and description loss during training (ablation) | ProDVa · Mol-Instructions validation set | performance across metrics | Helped |
| Increase either type-loss or description-loss weight continuously (ablation) | ProDVa · Mol-Instructions validation set | pLDDT | Hurt |
| Retrieve more than 16 relevant descriptions (ablation) | ProDVa · Mol-Instructions | ProTrek Score | Hurt |
| Increase the supporting-document set size (ablation) | ProDVa · Mol-Instructions | language alignment | Helped |
Surprising findings
- Even random incorporation of protein fragments (Random+ (E)) substantially improves foldability over random amino-acid sequences.
- Retrieving more fragment candidates (larger K) does not consistently help; pLDDT peaks and ProTrek Score declines beyond K=16 on Mol-Instructions.
Limitations
- Language alignment remains slightly below Pinal on ProTrek Score and EvoLlama Score.
- Fragments are limited to InterPro-annotated regions, so proteins without annotations contribute no fragments.
- Evaluation relies on in-silico oracle models (ProTrek, EvoLlama) and ESMFold rather than wet-lab validation.
Experimental setup
Datasets: SwissProt CAMEO subset Mol-Instructions protein design SwissProtCLAP InterPro annotations
Benchmarks: CAMEO subset Mol-Instructions protein design
Metrics: PPL Repetitiveness pLDDT PAE ProTrek Score EvoLlama Score Keyword Recovery Retrieval Accuracy Sequence Diversity
Baselines: ProteinDT Pinal PAAG ESM3 Chroma Random (U) Random (E) Random+ (E)
Builds on: ProtGPT2 dynamic vocabulary InterPro ESM3 Pinal
Closest papers by meaning
- Elucidating the Design Space of Multimodal Protein Language Models (ICML 2025 · similarity 0.89)
- Learning to engineer protein flexibility (ICLR 2025 · similarity 0.89)
- SurfPro: Functional Protein Design Based on Continuous Surface (ICML 2024 · similarity 0.88)
- ProtInvTree: Deliberate Protein Inverse Folding with Reward-guided Tree Search (NeurIPS 2025 · similarity 0.88)
- Steering Protein Language Models (ICML 2025 · similarity 0.88)
- Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.