Research map / Probabilistic and scientific modeling / Molecules and proteins
Contrastive losses as generalized models of global epistasis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that minimizing a Bradley-Terry contrastive ranking loss recovers sparse latent fitness functions under global epistasis and outperforms MSE on FLIP benchmark tasks, justified by a fitness-epistasis uncertainty principle.
Problem
Learning fitness functions from limited data is hard when global epistasis (a monotonic nonlinearity applied to a sparse latent function) makes the observed fitness dense in the epistatic domain, which hurts MSE-trained models.
Approach
Treat the latent fitness function under global epistasis as a ranking function and fit it with a pairwise Bradley-Terry contrastive loss that uses only label order, making it invariant to the monotonic nonlinearity and robust to epistatic density, supported by an entropic fitness-epistasis uncertainty principle.
Key results
- On FLIP, Bradley-Terry loss gives higher Spearman than MSE on most splits, e.g. GB1 3-vs-rest 0.880 vs 0.814 and Thermostability Mixed 0.453 vs 0.349.
- In simulations with 256 training points, MSE-trained models degrade as epistatic entropy rises while BT-trained models often reach near-perfect Spearman.
- BT loss recovers the sparse latent NK fitness function almost exactly from complete data under exponential global epistasis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · GB1 1-vs-rest | Spearman: 0.133 \pm 0.150 -> 0.091 \pm 0.093 | Hurt |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · GB1 2-vs-rest | Spearman: 0.564 \pm 0.026 -> 0.607 \pm 0.009 | Helped |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · GB1 3-vs-rest | Spearman: 0.814 \pm 0.049 -> 0.880 \pm 0.003 | Helped |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · AAV Mut-Des | Spearman: 0.751 \pm 0.006 -> 0.757 \pm 0.007 | No clear effect |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · AAV 1-vs-rest | Spearman: 0.335 \pm 0.117 -> 0.485 \pm 0.078 | Helped |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · Thermostability Mixed | Spearman: 0.349 \pm 0.011 -> 0.453 \pm 0.007 | Helped |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · Thermostability Human | Spearman: 0.511\pm 0.016 -> 0.589 \pm 0.002 | Helped |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · CAPSD_AAV2S_Sinai_2021 | Spearman: 0.912 \pm 0.003 -> 0.920 \pm 0.003 | Helped |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · GFP_AEQVI_Sarkisyan_2016 | Spearman: 0.867 \pm 0.001 -> 0.873 \pm 0.001 | Helped |
| Train the CNN with Bradley-Terry loss instead of MSE loss | CNN · FLIP tasks, training-set-size simulation | Spearman | Helped |
| Include unmodified sequence positions in the CNN input | CNN fitness prediction · GB1 Low-vs-High, GB1 3-vs-rest, and AAV 1-vs-rest | Spearman | Helped |
| Include unmodified sequence positions in the CNN input | CNN fitness prediction · FLIP tasks generally | Spearman | Mixed |
Surprising findings
- BT loss achieves higher Spearman than MSE even though Pearson correlation is worse, indicating ranking-based training is not simply favored by the metric.
- On a toy noisy-label scenario, the MSE model slightly outperforms the BT loss in AUC (0.977 vs 0.932).
Limitations
- No theoretical explanation of why BT loss is robust to dense epistatic representations.
- Analogy to compressed sensing scaling laws does not strictly apply to neural networks trained with SGD.
- Effects of complex experimental count noise are not analyzed; BT ignores gap magnitude between labels.
Experimental setup
Datasets: NK model simulated fitness landscapes FLIP benchmark (GB1, AAV, Thermostability) ProteinGym CAPSD_AAV2S_Sinai_2021 ProteinGym GFP_AEQVI_Sarkisyan_2016
Benchmarks: FLIP GB1 FLIP AAV FLIP Thermostability ProteinGym
Metrics: Spearman correlation Top 10% recall Pearson correlation
Baselines: MSE loss Margin ranking loss
Builds on: Bradley-Terry model NK model Graph Fourier transform compressed sensing FLIP benchmark
Closest papers by meaning
- A Variational Perspective on Generative Protein Fitness Optimization (ICML 2025 · similarity 0.87)
- DePLM: Denoising Protein Language Models for Property Optimization (NeurIPS 2024 · similarity 0.87)
- Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.87)
- Evolution-Inspired Loss Functions for Protein Representation Learning (ICML 2024 · similarity 0.86)
- Robust Model-Based Optimization for Challenging Fitness Landscapes (ICLR 2024 · similarity 0.85)
- Multi-Scale Representation Learning for Protein Fitness Prediction (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.