Research map / Label-efficient and robust learning / Self-supervised and contrastive
Evolution-Inspired Loss Functions for Protein Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EvoRank, an SSL loss that trains a structure-based protein model to rank amino acids according to MSA-derived empirical distributions, improving zero-shot mutation effect prediction over wildtype-masked training.
Problem
Wildtype-accuracy masked pretraining for protein representations does not align with mutation effect prediction for protein engineering; improved WT accuracy can hurt downstream zero-shot performance.
Approach
Replace the wildtype one-hot target with an MSA-derived soft label (initialization) and then train a pairwise ranking regression head that predicts the relative MSA likelihood of a "to" versus "from" amino acid in a masked microenvironment (EvoRank loss); MSAs are used only at training time.
Key results
- MutRank improves average zero-shot Pearson correlation by ~66% and AUC by ~16% over MutComputeXGT on ∆∆G datasets (T2837 etc.).
- On PlatinumDB, MutRank raises Pearson from 0.05 to 0.28 and AUC from 0.48 to 0.64 relative to MutComputeXGT.
- With only ~9% of cDNA data for fine-tuning, EvoRank representations outperform WT-mask representations by 16% Pearson, 15% Spearman, 7% AUC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace WT-mask pretraining with MSA soft-label training (ablation) | MutComputeXGT · FireProtDB | Pearson correlation: 0.30 -> 0.36 | Helped |
| Add EvoRank training after MSA soft-label training (ablation) | MutComputeXGT with MSA soft-label pretraining · T2837 | Pearson correlation: 0.37 -> 0.51 | Helped |
| Add EvoRank training after MSA soft-label training (ablation) | MutComputeXGT with MSA soft-label pretraining · S669 | Pearson correlation: 0.30 -> 0.40 | Helped |
| Add EvoRank training after MSA soft-label training (ablation) | MutComputeXGT with MSA soft-label pretraining · S-Sym | Pearson correlation: 0.48 -> 0.62 | Helped |
| Add EvoRank training after MSA soft-label training (ablation) | MutComputeXGT with MSA soft-label pretraining · Myoglobin | Pearson correlation: 0.45 -> 0.68 | Helped |
| Add EvoRank training after MSA soft-label training (ablation) | MutComputeXGT with MSA soft-label pretraining · FireProtDB | Pearson correlation: 0.36 -> 0.51 | Helped |
| Add EvoRank training after MSA soft-label training (ablation) | MutComputeXGT with MSA soft-label pretraining · Gβ1 | Pearson correlation: 0.41 -> 0.62 | Helped |
| Add EvoRank training after MSA soft-label training (ablation) | MutComputeXGT with MSA soft-label pretraining · T2837 Reverse | Pearson correlation: 0.37 -> 0.51 | Helped |
| Replace WT one-hot labels with MSA soft labels (ablation) | WT-mask training · Mask prediction test dataset, top-5 amino acids | Spearman correlation: 0.54 -> 0.60 | Helped |
| Replace WT one-hot labels with MSA soft labels (ablation) | WT-mask training · Mask prediction test dataset, top-10 amino acids | Spearman correlation: 0.38 -> 0.52 | Helped |
| Replace WT one-hot labels with MSA soft labels (ablation) | WT-mask training · Mask prediction test dataset, top-20 amino acids | Spearman correlation: 0.28 -> 0.34 | Helped |
| Use reverse KL divergence instead of KL divergence (ablation) | MSA soft-label training · Mask prediction test dataset, top-5 amino acids | Spearman correlation: 0.60 -> 0.54 | Hurt |
Surprising findings
- Downstream stability prediction performance decreases beyond about 68% wildtype accuracy for WT-mask trained models.
- The raw MSA log-odds baseline (Pearson 0.15 on cDNA117K) is far below MutRank (0.45), indicating the model generalizes beyond the MSA prior it was trained on.
Limitations
- Only validated on the microenvironment (structure-based) modality; not applied to ProteinMPNN or ESM2 due to architecture and compute constraints.
- Binding and solubility phenotype evaluations are limited, and the authors note more thorough evaluation is needed.
- Solubility DMS measured on chimeric variants, so interpretation is caveated.
Experimental setup
Datasets: MutComputeX training set (RCSB structures) cDNA117K UniRef50/UniRef90 MSAs
Benchmarks: T2837 S669 S-Sym Myoglobin FireProtDB Gβ1 SKEMPI-v2 S487 AB-Bind PlatinumDB levoglucosan kinase solubility DMS TEM1-β-lactamase solubility DMS
Metrics: Pearson correlation Spearman correlation AUC RMSE
Baselines: ESM2 ProteinMPNN MutComputeXGT Stability Oracle RaSP ThermoMPNN Prostata-IFML
Builds on: MutComputeX Stability Oracle WT-mask prediction learning-to-rank
Closest papers by meaning
- Tree Search-Based Evolutionary Bandits for Protein Sequence Optimization (AAAI 2024 · similarity 0.88)
- ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods (NeurIPS 2025 · similarity 0.88)
- DePLM: Denoising Protein Language Models for Property Optimization (NeurIPS 2024 · similarity 0.87)
- A Variational Perspective on Generative Protein Fitness Optimization (ICML 2025 · similarity 0.87)
- Multi-Scale Representation Learning for Protein Fitness Prediction (NeurIPS 2024 · similarity 0.87)
- Ultrafast classical phylogenetic method beats large protein language models on variant effect prediction (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.