Research map / Probabilistic and scientific modeling / Molecules and proteins
DePLM: Denoising Protein Language Models for Property Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DePLM, which denoises evolutionary likelihoods from protein language models via a rank-space diffusion process trained to maximize Spearman correlation, for mutation effect prediction and generalization to novel proteins.
Problem
Protein optimization relies on evolutionary information from PLMs, but this information mixes many properties and fine-tuning on experimental data overfits to dataset-specific measurements, hurting generalization to new proteins.
Approach
Treat PLM evolutionary likelihoods as property-relevant signal plus additive noise, and denoise them with a diffusion process defined in the rank space of property values, with a learning objective that maximizes Spearman rank correlation rather than minimizing numerical error.
Key results
- DePLM (ESM2) reaches 0.897 average Spearman on ProteinGym stability/fitness-style random cross-validation versus 0.904 for ProteinNPT's stability; it achieves 0.707 on fitness, above ProteinNPT's 0.668.
- In the generalization setting DePLM (ESM2) reaches 0.773 on stability and 0.518 on activity, outperforming SaProt (FT.) at 0.703 and 0.495.
- Ablation shows removing the rank objective drops average performance from 0.672 to 0.487, the largest drop among components.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DePLM (ESM2) rather than ProteinNPT | Protein fitness prediction · ProteinGym | Spearman correlation: 0.904 -> 0.897 | Hurt |
| Use DePLM (ESM2) rather than ProtSSN | Protein fitness prediction · β-lact., GB1, Fluo. | Spearman correlation: 0.692 -> 0.707; 0.718 -> 0.742; 0.757 -> 0.764 | Helped |
| Use DePLM (ESM2) rather than the strongest listed baseline | Protein fitness prediction · ProteinGym: Fitness, Expression, Binding, Activity | Spearman correlation: 0.823 -> 0.904; 0.533 -> 0.665; 0.536 -> 0.665; 0.600 -> 0.662 | Helped |
| Use DePLM (ESM2) rather than the strongest listed baseline | Generalization across proteins · ProteinGym: Stability, Fitness, Expression, Binding, Activity | Spearman correlation: 0.703 -> 0.773; 0.477 -> 0.480; 0.496 -> 0.510; 0.421 -> 0.441; 0.495 -> 0.518 | Helped |
| Add structural information (ablation) | DePLM · ProteinGym (Valid) | Spearman correlation: 0.683 -> 0.690 | Helped |
| Add structural information (ablation) | DePLM · GB1 | Spearman correlation: 0.672 -> 0.665 | Hurt |
| Add structural information (ablation) | DePLM · Fluorescence | Spearman correlation: 0.659 -> 0.662 | Helped |
| Add feature encoder (ablation) | DePLM · ProteinGym (Valid), GB1, Fluorescence | Spearman correlation: 0.682 -> 0.690; 0.659 -> 0.665; 0.661 -> 0.662 | Helped |
| Add denoising block (ablation) | DePLM · ProteinGym (Valid), GB1, Fluorescence | Spearman correlation: 0.656 -> 0.690; 0.644 -> 0.665; 0.655 -> 0.662 | Helped |
| Use the rank objective (ablation) | DePLM · ProteinGym (Valid), GB1, Fluorescence | Spearman correlation: 0.322 -> 0.690; 0.588 -> 0.665; 0.552 -> 0.662 | Helped |
| Increase diffusion steps beyond 3 (ablation) | DePLM · Testing data | Spearman correlation: 0.716 -> 0.685; 0.716 -> 0.587; 0.716 -> 0.575; 0.716 -> 0.576; 0.716 -> 0.567 | Hurt |
Surprising findings
- Performance does not monotonically improve with more diffusion steps; the best result uses 3 steps, and test Spearman falls at higher step counts, attributed to overfitting.
- Substituting the denoising module with plain MLPs (zero diffusion steps) consistently lowers performance.
Limitations
- Experiments use wild-type marginal probability, which sums individual mutation effects and ignores epistatic interactions between mutations.
- Total parameter count is large (about 834M including frozen PLM), with 42.2M trainable.
- Performance drops with more diffusion steps due to overfitting, so the number of steps must be tuned.
Experimental setup
Datasets: ProteinGym PEER beta-lactamase PEER fluorescence FLIP GB1
Benchmarks: ProteinGym beta-lactamase Fluorescence GB1
Metrics: Spearman's rank correlation
Baselines: CNN ResNet LSTM Transformer OHE ESM-MSA Tranception ProteinNPT ESM-2 ProtSSN SaProt ProteinMPNN
Builds on: ESM-1v ESM-IF ProteinNPT denoising diffusion models Quicksort
Closest papers by meaning
- Diffusion Language Models Are Versatile Protein Learners (ICML 2024 · similarity 0.90)
- DPLM-2: A Multimodal Diffusion Protein Language Model (ICLR 2025 · similarity 0.90)
- Knowledge-aware Reinforced Language Models for Protein Directed Evolution (ICML 2024 · similarity 0.89)
- Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.89)
- Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Tasks (ICML 2025 · similarity 0.89)
- Metalic: Meta-Learning In-Context with Protein Language Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.