Research map / Label-efficient and robust learning / Self-supervised and contrastive

Multi-Scale Representation Learning for Protein Fitness Prediction

NeurIPS 2024 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Self-supervision in supervised pipelines · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes S2F and S3F, multi-scale zero-shot fitness predictors that add GVP-based structure and surface (point cloud) encoders on top of frozen ESM-2-650M embeddings, pre-trained on CATH, and evaluates them on 217 ProteinGym substitution assays.

Problem

Zero-shot prediction of protein mutation effects (fitness landscapes) is limited by scarce functional labels, and existing sequence-structure hybrids give only incremental gains over sequence-only models while ignoring fine-grained surface topology.

Approach

Stack geometric vector perceptron message passing on a backbone graph and a surface point-cloud graph, initialized with frozen ESM-2 residue embeddings, and pre-train with masked residue-type prediction on CATH so the model yields a residue-type distribution used as a zero-shot mutation score.

Key results

• S3F achieves Spearman 0.470 on ProteinGym (217 assays) without MSAs, the best in its category, versus 0.457 for SaProt and 0.454 for S2F.
• S3F-MSA (ensembled with EVE z-scores) reaches Spearman 0.496, an 8.5% improvement over SaProt-based state of the art cited by the authors.
• S3F has about 20M trainable parameters versus 650M for SaProt; pre-training takes 9 hours for S2F and 58 hours for S3F on four A100 GPUs.

Measured effects reported in the paper

Surprising findings

• Removing structure while keeping surface features (S3F w/o structure, 0.392) degrades performance below the sequence-only ESM2 baseline (0.414).
• Performance of all methods, including sequence-only ESM, correlates with AlphaFold2 structure quality, suggesting predicted-structure reliance.

Limitations

• Pre-trained on a relatively small set of ~31K experimentally determined CATH structures rather than larger predicted-structure databases.
• Assumes backbone is unchanged by mutations and omits side-chain information.
• Handles only substitutions, not insertions or deletions.

Experimental setup

Datasets: CATH v4.3.0 (non-redundant subset) ProteinGym (217 substitution DMS assays)

Benchmarks: ProteinGym

Metrics: Spearman AUC MCC NDCG Top 10% Recall

Baselines: ProGen2 XL CARP ESM-2-650M ProteinMPNN MIF ESM-IF MIF-ST ProtSSN SaProt DeepSequence EVE GEMME

Builds on: ESM-2 Geometric Vector Perceptron dMaSIF ProteinGym

Closest papers by meaning

• Metalic: Meta-Learning In-Context with Protein Language Models (ICLR 2025 · similarity 0.89)
• Evaluating Representation Learning on the Protein Structure Universe (ICLR 2024 · similarity 0.88)
• Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.87)
• Understanding protein function with a multimodal retrieval-augmented foundation model (NeurIPS 2025 · similarity 0.87)
• DPLM-2: A Multimodal Diffusion Protein Language Model (ICLR 2025 · similarity 0.87)
• ProtCLIP: Function-Informed Protein Multi-Modal Learning (AAAI 2025 · similarity 0.87)