Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Towards precision protein-ligand affinity prediction benchmark: A Complete and Modification-Aware DAVIS Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Curates DAVIS-Complete by adding 4,032 modified kinase–ligand pairs (substitutions, insertions, deletions, phosphorylation) and benchmarks docking-free and docking-based affinity models under three modification-aware settings.
Problem
Binding affinity models are trained and evaluated on DAVIS datasets that either ignore or discard protein modifications (mutations, deletions, insertions, PTMs), so they overfit to wild-type proteins and their generalization to modified proteins is unknown.
Approach
Extend DAVIS with manually curated modified kinase sequences and domain-refined entries, then define three benchmarks (augmented dataset prediction, wild-type to modification zero-shot generalization, and few-shot modification fine-tuning) that test modification awareness of docking-free and docking-based models.
Key results
- Boltz-2 outperforms FDA across most splits, e.g. Ligand-structure & Protein-seqid Modification subset with delta MSE -0.42 and delta Rp +0.29, but on the complete test set DeepDTA ranks above Boltz-2 on some splits.
- Docking-free models trained only on wild-type pairs fall to Rp near 0.1 on same-ligand different-modification pairs (DeepDTA Rp 0.10), while Boltz-2 reaches 0.29; these models often fail to beat the wild-type ground-truth baseline.
- After few-shot fine-tuning on same-modification different-ligand data, AttentionDTA MSE drops from 0.62 to 0.42 and Rp rises from 0.77 to 0.80, whereas FDA and Boltz-2 do not benefit.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Boltz-2 rather than FDA | docking-based affinity prediction · Augmented Dataset Prediction, ligand-structure & protein-seqid | MSE: 0.89 (0.13) -> 0.62 (0.07) | Helped |
| Use Boltz-2 rather than FDA | docking-based affinity prediction · Augmented Dataset Prediction, ligand-structure & protein-seqid | R_p: 0.28 (0.07) -> 0.58 (0.07) | Helped |
| Use docking-based prediction rather than docking-free prediction | Augmented Dataset Prediction · New-ligand and both-new splits, excluding protein-modification and protein-name | R_p | Helped |
| Train and test with protein-modification as the new-protein split | Augmented Dataset Prediction models · Protein-modification split | MSE: MSE < 0.5 | Helped |
| Use the stricter ligand-structure split instead of ligand-name | Augmented Dataset Prediction models · New-ligand splits | R_p | Hurt |
| Use the stricter protein-seqid split instead of protein-modification or protein-name | Augmented Dataset Prediction models · New-protein splits | R_p | Hurt |
| Train only on wild-type pairs and evaluate modified pairs | docking-free models · Wild-Type to Modification, same-ligand, different-modifications | R_p: below 0.3 | Hurt |
| Fine-tune on limited modified protein-ligand pairs | docking-free models · Few-Shot Modification, same-ligand, different-modifications | MSE: DeepDTA: 0.62 (0.73) -> 0.33 (0.43); AttentionDTA: 0.68 (0.92) -> 0.34 (0.45); GraphDTA: 1.32 (2.09) -> 0.83 (1.18); DGr | Helped |
| Fine-tune on limited modified protein-ligand pairs | FDA · Few-Shot Modification, same-modification, different-ligands | MSE: 1.15 (1.09) -> 2.27 (1.62) | Hurt |
| Fine-tune on limited modified protein-ligand pairs | Boltz-2 · Few-Shot Modification, same-modification, different-ligands | MSE: 0.70 (0.47) -> 0.62 (0.48) | Hurt |
| Fine-tune on limited modified protein-ligand pairs | AttentionDTA · Few-Shot Modification, same-modification, different-ligands | MSE: 0.62 (0.75) -> 0.42 (0.45) | Helped |
| Fine-tune on limited modified protein-ligand pairs | AttentionDTA · Few-Shot Modification, same-modification, different-ligands | R_p: 0.77 (0.26) -> 0.80 (0.24) | Helped |
Surprising findings
- Docking-free models (e.g., DeepDTA) outperform docking-based Boltz-2 on the complete test set and modification subset in the augmented dataset prediction benchmark.
- AlphaFold3 predicts a phosphorylated state for both phosphorylated and non-phosphorylated ABL1, failing to distinguish them.
Limitations
- About 70% of DAVIS Kd values are capped at 10 uM, so modification-induced affinity changes are often only lower bounds or untrackable.
- Possible data leakage between DAVIS-Complete and training data of AlphaFold-Multimer, DiffDock, and Boltz-2, which may overstate docking-based performance.
- Dataset is small and restricted to kinases, limiting generalization to the broader proteome.
Experimental setup
Datasets: DAVIS AlphaFold Protein Structure Database UniProt AlphaFold-Multimer training data DiffDock training data PDB MISATO ATLAS mdCATH
Benchmarks: DAVIS-Complete augmented dataset prediction DAVIS-Complete wild-type to modification generalization DAVIS-Complete few-shot modification generalization
Metrics: MSE Pearson correlation C-index
Baselines: DeepDTA AttentionDTA GraphDTA DGraphDTA MGraphDTA Folding-Docking-Affinity Boltz-2
Builds on: DAVIS Folding-Docking-Affinity Boltz-2 DeepDTA GraphDTA
Closest papers by meaning
- UniSite: The First Cross-Structure Dataset and Learning Framework for End-to-End Ligand Binding Site Detection (NeurIPS 2025 · similarity 0.87)
- DecoyDB: A Dataset for Graph Contrastive Learning in Protein-Ligand Binding Affinity Prediction (NeurIPS 2025 · similarity 0.87)
- Deep Confident Steps to New Pockets: Strategies for Docking Generalization (ICLR 2024 · similarity 0.87)
- Rethinking the generalization of drug target affinity prediction algorithms via similarity aware evaluation (ICLR 2025 · similarity 0.86)
- Knowledge Enhanced Representation Learning for Drug Discovery (AAAI 2024 · similarity 0.86)
- M²N: A Progressive Macro-to-Micro 3D Modeling Scheme for Unveiling Drug-Target Affinity (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.