Research map / Probabilistic and scientific modeling / Molecules and proteins
Tranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-time Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add inference-time retrieval of homologous sequences (ablation) | Tranception · ProteinGym substitution benchmark, all MSA depths | Spearman: 0.406 -> 0.451 | Helped |
| Add inference-time retrieval of homologous sequences (ablation) | Tranception · ProteinGym indel benchmark | Spearman: 0.430 -> 0.463 | Helped |
| Use Tranception's grouped, multi-kernel attention instead of GPT2 attention (ablation) | Tranception S · ProteinGym substitution benchmark, validation set | Spearman: 0.324 -> 0.344 | Helped |
| Use Tranception's grouped, multi-kernel attention instead of Primer attention (ablation) | Tranception S · ProteinGym substitution benchmark, validation set | Spearman: 0.314 -> 0.344 | Helped |
| Use Grouped ALiBi instead of learned position embeddings (ablation) | Tranception · ProteinGym substitution benchmark, full set | Spearman: 0.333 -> 0.335 | Helped |
| Train on UniRef100 rather than UniRef90 (ablation) | Tranception S · ProteinGym substitution benchmark, full set | Spearman: 0.275 -> 0.335 | Helped |
| Train on UniRef100 rather than UniRef50 (ablation) | Tranception S · ProteinGym substitution benchmark, full set | Spearman: 0.247 -> 0.335 | Helped |
| Scale from the small to the large Tranception model (ablation) | Tranception · ProteinGym substitution benchmark, full set | Spearman: 0.335 -> 0.404 | Helped |
| Average scores from forward and reversed sequence traversal (ablation) | Tranception L · ProteinGym substitution benchmark, full set | Spearman: 0.376 -> 0.401 | Helped |
| Increase retrieval weight from 0.0 to 0.6 (ablation) | Tranception L · ProteinGym substitution benchmark, validation set | Spearman: 0.399 -> 0.452 | Helped |
| Use retrieval-only scoring at covered positions instead of mixing retrieval with autoregressive scoring (ablation) | Tranception L · ProteinGym substitution benchmark, validation set | Spearman: 0.452 -> 0.397 | Hurt |
Closest papers by meaning
- Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.90)
- Understanding protein function with a multimodal retrieval-augmented foundation model (NeurIPS 2025 · similarity 0.89)
- Metalic: Meta-Learning In-Context with Protein Language Models (ICLR 2025 · similarity 0.88)
- A Variational Perspective on Generative Protein Fitness Optimization (ICML 2025 · similarity 0.88)
- Training Compute-Optimal Protein Language Models (NeurIPS 2024 · similarity 0.88)
- DePLM: Denoising Protein Language Models for Property Optimization (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.