Research map / Probabilistic and scientific modeling / Molecules and proteins
Controlling Repetition in Protein Language Models
Jiahao Zhang, ZEQING ZHANG, Di Wang, Lijie Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| inject utility-controlled contrastive steering vector at inference | ProtGPT2 · CATH unconditional generation | repetition score R: 0.728 -> 0.845 | Improved |
| inject utility-controlled contrastive steering vector at inference | ProtGPT2 · CATH unconditional generation | utility score U: 0.621 -> 0.711 | Improved |
| inject steering vector | ESM-3 · CATH unconditional generation | repetition score R: +55% | Improved |
| use temperature sampling to diversify output | ProtGPT2 · CATH unconditional generation | utility score U: 0.621 -> 0.612 | Hurt |
| use repetition penalty decoding | ProtGPT2 · CATH unconditional generation | repetition score R: 0.728 -> 0.780 | Improved |
| use top-p sampling | ProtGPT2 · CATH unconditional generation | repetition score R: 0.728 -> 0.714 | Hurt |
| apply neuron deactivation or probe steering baselines | ESM-3 and ProtGPT2 · unconditional and conditional generation | R and U | Mixed |
| build contrastive sets with Pareto or composite selection instead of random sampling (ablation) | UCCS · ESM3 and ProtGPT2 ablation | R and U | Improved |
| increase steering strength alpha beyond about 1-2 (ablation) | UCCS · ESM3 and ProtGPT2 ablation | R and U | Mixed |
| inject steering in later layers (ablation) | UCCS · ESM3 and ProtGPT2 layer ablation | R and U | Improved |
Closest papers by meaning
- Steering Protein Language Models (ICML 2025 · similarity 0.88)
- Elucidating the Design Space of Multimodal Protein Language Models (ICML 2025 · similarity 0.88)
- Protein Design with Dynamic Protein Vocabulary (NeurIPS 2025 · similarity 0.87)
- Learning to engineer protein flexibility (ICLR 2025 · similarity 0.87)
- Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.87)
- Bridge-IF: Learning Inverse Protein Folding with Markov Bridges (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
