Research map / Probabilistic and scientific modeling / Molecules and proteins
3DMolFormer: A Dual-channel Framework for Structure-based Drug Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3DMolFormer, a dual-channel GPT-2-style transformer that autoregressively generates parallel token and numerical sequences for pocket-ligand complexes, unifying protein-ligand docking (supervised fine-tuning) and pocket-aware 3D drug design (RL fine-tuning).
Problem
Structure-based drug discovery has two dual tasks, docking and pocket-aware 3D drug design, but no single ML method handles both; existing methods struggle to model continuous 3D coordinates with discrete token models and are limited by scarce ground-truth complex data.
Approach
Represent pocket-ligand complexes as parallel discrete token and continuous number sequences, and use a dual-channel transformer (token head plus number head) trained with CE+MSE loss; large-scale pre-training, then MSE fine-tuning for docking and regularized-MLE RL fine-tuning for design, reusing the docking weights to produce coordinates during design.
Key results
- On CASF-2016 docking, 3DMolFormer reaches 84.9% below 2.0 Å and average RMSD 1.29 Å, vs Uni-Mol 80.4% and 1.62 Å.
- On pocket-aware drug design, success rate 85.3% vs best baseline DecompDiff 24.5%, with Vina Dock -9.48 vs -8.39.
- RL fine-tuning is essential: w/o RL success rate drops to 2.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3DMolFormer instead of Uni-Mol | CASF-2016 docking · CASF-2016 | % RMSD <2.0 Å: 80.4 -> 84.9 | Helped |
| Use 3DMolFormer instead of Uni-Mol | CASF-2016 docking · CASF-2016 | average RMSD: 1.62 -> 1.29 | Helped |
| Use 3DMolFormer instead of Smina | CASF-2016 docking · CASF-2016 | % RMSD <1.0 Å: 47.4 -> 43.8 | Hurt |
| Use 3DMolFormer instead of DecompDiff | Pocket-aware 3D drug design · 100 CrossDocked2020 pockets | Success Rate: 24.5% -> 85.3% | Helped |
| Use 3DMolFormer instead of Pocket2Mol | Pocket-aware 3D drug design · 100 CrossDocked2020 pockets | QED: 0.56 -> 0.49 | Hurt |
| Add pre-training (ablation) | 3DMolFormer docking · CASF-2016 | % RMSD <2.0 Å: 57.8 -> 84.9 | Helped |
| Add pre-training (ablation) | 3DMolFormer docking · CASF-2016 | average RMSD: 2.25 -> 1.29 | Helped |
| Add data augmentation (ablation) | 3DMolFormer docking · CASF-2016 | % RMSD <2.0 Å: 51.0 -> 84.9 | Helped |
| Add data augmentation (ablation) | 3DMolFormer docking · CASF-2016 | average RMSD: 2.45 -> 1.29 | Helped |
| Fine-tune with RL rather than freezing GPT weights (ablation) | 3DMolFormer drug design · Pocket-aware 3D drug design | Success Rate: 2.1% -> 85.3% | Helped |
| Increase model scale to 12 layers, 12 heads, embedding length 768 (ablation) | 3DMolFormer pre-training · Ligand validation set | pre-training loss: 0.229 -> 0.178 | Helped |
| Change α from 0.1 to 1.0 (ablation) | 3DMolFormer pre-training · Pre-training | CE Loss: 0.164 -> 0.164 | No clear effect |
Surprising findings
- Without pre-training, the docking model has much lower sub-1 Å accuracy (15.5%) than with pre-training, and the model does not need an initialized ligand conformation as input.
- Smina outperforms all deep learning methods on the sub-1.0 Å RMSD percentage (47.4%), while 3DMolFormer leads on the other thresholds.
Limitations
- Does not model protein flexibility, which may limit binding affinity accuracy.
- Does not capture binding dynamics or environmental factors such as temperature and pH.
- Sub-1 Å docking accuracy still trails search-based Smina.
Experimental setup
Datasets: Uni-Mol pocket and conformation data (3.2M pockets, 209M conformations) CrossDocked2020 PDBbind v2020 CASF-2016 PoseBusters
Benchmarks: CASF-2016 CrossDocked2020 100 pockets PoseBusters
Metrics: RMSD below thresholds average RMSD Vina Score Vina Dock QED SA success rate Delta Score clash score strain energy
Baselines: AutoDock4 AutoDock Vina Vinardo Smina Uni-Mol AR liGAN GraphBP Pocket2Mol TargetDiff DecompDiff
Builds on: GPT-2 xVal Uni-Mol regularized maximum likelihood estimation
Closest papers by meaning
- ShEPhERD: Diffusing shape, electrostatics, and pharmacophores for bioisosteric drug design (ICLR 2025 · similarity 0.88)
- TurboHopp: Accelerated Molecule Scaffold Hopping with Consistency Models (NeurIPS 2024 · similarity 0.88)
- M²N: A Progressive Macro-to-Micro 3D Modeling Scheme for Unveiling Drug-Target Affinity (AAAI 2025 · similarity 0.87)
- Controllable 3D Molecular Generation for Structure-Based Drug Design Through Bayesian Flow Networks and Gradient Integration (NeurIPS 2025 · similarity 0.87)
- CBGBench: Fill in the Blank of Protein-Molecule Complex Binding Graph (ICLR 2025 · similarity 0.87)
- Manipulating 3D Molecules in a Fixed-Dimensional E(3)-Equivariant Latent Space (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.