Research map / Probabilistic and scientific modeling / Molecules and proteins
Augmenting Biological Fitness Prediction Benchmarks with Landscapes Features from GraphFLA
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GraphFLA, a Python framework that constructs sequence-fitness landscapes from mutagenesis data and computes 20 topography features to help interpret and compare fitness prediction model performance, and releases 155 combinatorially complete empirical landscapes.
Problem
Fitness prediction benchmarks (ProteinGym, RNAGym) report averaged scores and lack landscape topography information, making it hard to explain why models perform differently across tasks or why one model beats another on some tasks.
Approach
Construct each mutational fitness landscape as a directed variant graph with scalable single-mutant neighbor generation, compute 20 biologically grounded topography features (ruggedness, epistasis, navigability, neutrality), and use them as meta-features to analyze benchmark model performance.
Key results
- GraphFLA constructed a 1-million-mutant landscape in about 20 s and 2 GB memory, whereas baselines took over 3-5 h and the community implementation ran out of memory above 100,000 mutants.
- Evo2-7b performance on the 155 combinatorial landscapes correlates strongly with landscape features, with half of features giving |rho| above 0.6.
- On ProteinGym, ProSST outperforms VenusREM on benign landscapes with reciprocal sign epistasis below 0.1 (Wilcoxon p=0.003), while VenusREM wins when it exceeds 0.15.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GraphFLA instead of existing landscape-construction implementations | NK landscapes · NK landscape, 1 million mutants | construction time: 20s; more than 5h; 3h | Helped |
| Use GraphFLA for landscape construction | NK landscapes · NK landscape, 1 million mutants | memory usage: 2GB | Helped |
| Remove a fraction of variants | GraphFLA feature calculations · NK landscape (n=15, k=7) | global optima accessibility: 0.6729 -> 0.6103; 0.5276; 0.3223 | Hurt |
| Remove a fraction of variants | GraphFLA feature calculations · NK landscape (n=15, k=7) | reciprocal sign epistasis: 0.1885 -> 0.1883; 0.1884; 0.1774 | No clear effect |
| Add Gaussian fitness noise up to 0.2σ | GraphFLA feature calculations · NK landscape (n=15, k=7) | global optima accessibility: 0.6729 -> 0.6339 | No clear effect |
| Use biased random-mutagenesis sampling | GraphFLA feature calculations · NK landscape (n=15, k=7) | reciprocal sign epistasis: 0.1885 -> 0.1823 | No clear effect |
| Increase landscape ruggedness and epistasis | Evo2-7b · 155 combinatorially complete landscapes | Spearman's ρ | Hurt |
| Decrease landscape navigability | Evo2-7b · 155 combinatorially complete landscapes | Spearman's ρ | Hurt |
| Increase landscape neutrality | Evo2-7b · 155 combinatorially complete landscapes | Spearman's ρ | Hurt |
| Apply ProSST instead of VenusREM on landscapes with εreci < 0.1 | ProteinGym zero-shot prediction · ProteinGym landscapes with εreci < 0.1 | Spearman's ρ: w = 123, p = 0.003 | Helped |
| Apply VenusREM instead of ProSST on landscapes with εreci > 0.15 | ProteinGym zero-shot prediction · ProteinGym landscapes with εreci > 0.15 | Spearman's ρ | Helped |
| Use supervised Kermut rather than zero-shot VenusREM | ProteinGym prediction on less navigable landscapes · ODP2 landscape | Spearman's ρ: 0.53 | Helped |
Surprising findings
- Zero-shot models VenusREM and ProSST only beat the supervised Kermut on highly navigable landscapes (FDC around -0.7), and Kermut outperforms VenusREM by 0.53 Spearman rho on the ODP2 landscape with FDC=0.23.
Limitations
- Visualizing high-dimensional fitness landscapes is difficult, and dimensionality reduction can distort local optima.
- Analyzable landscape sizes remain small relative to the full genotype space.
- RNAGym data and benchmark scores cannot be shared without permission from original authors.
Experimental setup
Datasets: ProteinGym DMS substitution tasks RNAGym fitness tasks CIS-BP PBM data 155 combinatorially complete landscapes collected from literature
Benchmarks: ProteinGym RNAGym CIS-BP 155 combinatorially complete landscapes
Metrics: Spearman's rho runtime peak memory Wilcoxon signed-rank test
Baselines: MAGELLAN community NK landscape implementation
Builds on: ProteinGym RNAGym MAGELLAN NK model flacco
Closest papers by meaning
- Improving protein optimization with smoothed fitness landscapes (ICLR 2024 · similarity 0.88)
- Multi-Scale Representation Learning for Protein Fitness Prediction (NeurIPS 2024 · similarity 0.87)
- Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.86)
- Robust Model-Based Optimization for Challenging Fitness Landscapes (ICLR 2024 · similarity 0.85)
- A Variational Perspective on Generative Protein Fitness Optimization (ICML 2025 · similarity 0.85)
- Contrastive losses as generalized models of global epistasis (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.