Research map / Probabilistic and scientific modeling / Molecules and proteins
Protriever: End-to-End Differentiable Protein Homology Search for Fitness Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Protriever jointly trains an ESM-2-initialized dense retriever and a PoET reader end-to-end (via EMDR loss) so the retriever learns which homologs help zero-shot protein fitness prediction, with vector search ~100x faster than MMseqs2-GPU.
Problem
MSA-based homolog retrieval for protein fitness prediction is computationally expensive, struggles with divergent sequences and indels, and is decoupled from the downstream modeling objective.
Approach
Make homolog retrieval a learned dense vector search over a UniRef50 index and backpropagate the reader's sequence-modeling likelihood (EMDR multi-document loss, reader stop-gradient) into the retriever, so retrieval is optimized jointly with the protein language model.
Key results
- Protriever reaches Spearman 0.479 on ProteinGym substitutions, above PoET (0.470) and the other sequence-based baselines.
- Retrieval takes about 0.0046 s per query with Faiss versus 0.613 s for MMseqs2-GPU and 2501 s for JackHMMER, per the paper's benchmark.
- Ablation: frozen ESM retriever 0.432, frozen DPR 0.440, Protriever without DPR 0.466, full Protriever 0.479 Spearman.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| jointly train retrieval and reader with DPR initialization | Protriever · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.432 -> 0.479 | Helped |
| add DPR initialization to end-to-end joint training (ablation) | Protriever · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.466 -> 0.479 | Helped |
| replace frozen ESM retriever with DPR (ablation) | frozen retriever with PoET reader · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.432 -> 0.440 | Helped |
| jointly train retriever and reader without DPR initialization (ablation) | Protriever · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.432 -> 0.466 | Helped |
| use Protriever retrieval instead of MMseqs2 (k-mer) | PoET fitness prediction · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.463 -> 0.479 | Helped |
| use Protriever retrieval instead of MMseqs2-GPU | PoET fitness prediction · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.462 -> 0.479 | Helped |
| use Protriever retrieval instead of JackHMMER | PoET fitness prediction · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.459 -> 0.479 | Helped |
| replace frozen Protriever retriever with trained Protriever retriever (ablation) | FiD reader · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.342 -> 0.416 | Helped |
| use FiD with trained Protriever rather than FiD with MSA | FiD reader · ProteinGym substitution DMS, average across MSA depths | Spearman: 0.420 -> 0.416 | Hurt |
Surprising findings
- JackHMMER retrieves sequences with higher identity to the query yet gives slightly lower fitness prediction Spearman than MMseqs2 and Protriever.
Limitations
- Index is only refreshed 10 times during training, so embeddings can go stale relative to the retriever.
- Fitness results are on the ProteinGym substitution benchmark only; improvements over PoET are small (0.479 vs 0.470).
- Preliminary FiD results were not tuned and DPR pretraining was not applied to them.
Experimental setup
Datasets: UniRef50 UniRef100 UniRef90 ProteinGym substitution DMS benchmark
Benchmarks: ProteinGym substitution benchmark (217 DMS assays)
Metrics: Spearman correlation AUC MCC top-K recall NDCG retrieval time per query
Baselines: EVE GEMME Site independent ESM-1v ProGen2 ESM2 MSA Transformer Tranception L TranceptEVE L PoET MMseqs2 MMseqs2-GPU
Builds on: PoET EMDR Dense Passage Retrieval ESM-2 Faiss
Closest papers by meaning
- ProSST: Protein Language Modeling with Quantized Structure and Disentangled Attention (NeurIPS 2024 · similarity 0.85)
- LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling (NeurIPS 2026 · similarity 0.85)
- Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment (NeurIPS 2025 · similarity 0.85)
- ProtInvTree: Deliberate Protein Inverse Folding with Reward-guided Tree Search (NeurIPS 2025 · similarity 0.85)
- ProtCLIP: Function-Informed Protein Multi-Modal Learning (AAAI 2025 · similarity 0.84)
- ProtGO: Function-Guided Protein Modeling for Unified Representation Learning (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.