Research map / Probabilistic and scientific modeling / Molecules and proteins

Ultrafast classical phylogenetic method beats large protein language models on variant effect prediction

NeurIPS 2024 · Probabilistic and scientific modeling › Molecules and proteins · approach: Protein fitness and variant effect prediction · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FastCherries, a near-linear-time divide-and-conquer cherry reconstruction that plugs into CherryML to estimate LG-type and site-specific (SiteRM) substitution rate matrices from MSAs, and SiteRM beats ESM-1v and EVmutation on ProteinGym DMS variant effect prediction.

Problem

Estimating amino acid substitution rate matrices from large MSAs requires expensive phylogenetic tree reconstruction, which is a computational bottleneck; the paper also asks whether site-specific evolutionary models can predict variant effects well.

Approach

Replace full tree estimation with a near-linear divide-and-conquer pairing of sequences into cherries (minimizing Hamming distance), estimate site rates and cherry distances by coordinate ascent, and then fit per-site rate matrices regularized toward the LG matrix; score variants by conditioning the evolutionary model on the wild-type at t=1.

Key results

• SiteRM achieves Spearman 0.423 and AUC 0.734 on ProteinGym DMS substitutions, outperforming ESM-1v ensemble (0.407 / 0.723) and EVmutation (0.395 / 0.716), though lower Recall@10 (0.207 vs 0.226 for DeepSequence).
• FastCherries processes an MSA with ~454K sequences of length 364 in about 1,000 seconds on one CPU core, with end-to-end SiteRM estimation around 1,500 seconds.
• CherryML with FastCherries is roughly 10-100x faster end-to-end than CherryML with FastTree, with about 50% lower statistical efficiency and ~2% asymptotic bias in simulations.

Measured effects reported in the paper

Surprising findings

• An independent-sites model (SiteRM) outperforms large protein language models such as ESM-1v and ESM-2 (15B) on DMS variant effect prediction.
• SiteRM scores on the human clinical benchmark (AUC 0.911) come within 0.01 of state of the art, despite not modeling epistasis.

Limitations

• Assumes time-reversible substitution models, which may cause model misspecification.
• Small asymptotic bias (~2% median relative error) and ~50% statistical efficiency loss versus ground-truth or FastTree trees; may yield less accurate tree topologies.
• Recall@10 on DMS benchmark is less competitive than some baselines (e.g., DeepSequence, GEMME).

Experimental setup

Datasets: ProteinGym MSAs CherryML simulated benchmark LG Pfam benchmark QMaker datasets TrRosetta dataset

Benchmarks: ProteinGym DMS substitutions ProteinGym human clinical substitutions CherryML simulated benchmark LG Pfam held-out likelihood QMaker datasets

Metrics: Spearman correlation AUC MCC NDCG@10 Recall@10 median relative error held-out log-likelihood runtime

Baselines: EVmutation DeepSequence ESM-1v ESM-IF1 ESM-1b ESM-2 (15B) TranceptEVE GEMME EVE MSA Transformer Tranception L VESPA

Builds on: CherryML LG model FastTree FastMG EVmutation

Closest papers by meaning

• LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling (NeurIPS 2026 · similarity 0.86)
• Evolution-Inspired Loss Functions for Protein Representation Learning (ICML 2024 · similarity 0.86)
• MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training (NeurIPS 2024 · similarity 0.86)
• MutaPLM: Protein Language Modeling for Mutation Explanation and Engineering (NeurIPS 2024 · similarity 0.86)
• DePLM: Denoising Protein Language Models for Property Optimization (NeurIPS 2024 · similarity 0.85)
• Training Compute-Optimal Protein Language Models (NeurIPS 2024 · similarity 0.85)