Research map / Large language models / Language, safety and interpretability
MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text Prompts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MolTailor, which uses a language model (PubMedBERT) as an agent over a molecular pretraining model knowledge base, trained on a new MT-MTR molecule-text multi-task regression dataset, to produce task-specific molecular representations from text prompts.
Problem
General-purpose molecular representations weight all features equally, so they ignore which properties matter for a specific downstream task, hurting training efficiency and predictive accuracy.
Approach
A dual-tower design where a language model (T-Encoder) reads a natural-language task description and, via cross-attention over the molecular encoder's hidden states, reweights molecular features; pretrained on MT-MTR, where text prompts name RDKit descriptors whose values are regression targets.
Key results
- MolTailor* (ChemBERTa-2 backbone) reaches RMSE 0.7128 on ESOL and 1.7826 on FreeSolv and 0.7848 on Lipophilicity, SOTA among compared methods in the linear probe setting.
- On classification, MolTailor with CHEM-BERT drops on BBBP (81.15 vs 84.10 ROC-AUC) and ClinTox (92.37 vs 93.80 for CHEM-BERT per Table 1 reporting), but gains on HIV (77.42) and Tox21 (80.67).
- Replacing GPT-4 task descriptions with irrelevant text ('to be or not to be') degrades regression RMSE from 0.8348 to 0.8541 on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use task-tailored representations with CHEM-BERT instead of the CHEM-BERT backbone representation | MolTailor · MoleculeNet regression tasks (ESOL, FreeSolv, Lipophilicity, QM8) | RMSE / MAE: 0.7973 -> 0.7234; 2.0214 -> 1.7881; 0.8571 -> 0.8107; 0.0215 -> 0.0196 | Helped |
| Use task-tailored representations with CHEM-BERT instead of the CHEM-BERT backbone representation | MolTailor · MoleculeNet classification tasks (BBBP, ClinTox, HIV, Tox21) | ROC-AUC: 84.10 -> 81.15; 93.80 -> 92.37; 76.99 -> 77.42; 80.54 -> 80.67 | Mixed |
| Use task-tailored representations with ChemBERTa-2 instead of the ChemBERTa-2 backbone representation | MolTailor* · MoleculeNet regression tasks | RMSE / MAE: 0.8103 -> 0.7128; 1.8439 -> 1.7826; 0.7948 -> 0.7848; 0.0191 -> 0.0185 | Helped |
| Use task-tailored representations with ChemBERTa-2 instead of the ChemBERTa-2 backbone representation | MolTailor* · MoleculeNet classification tasks | ROC-AUC: 84.70 -> 84.65; 84.21 -> 85.95; 78.88 -> 76.42; 80.75 -> 80.32 | Mixed |
| Pretrain on MT-MTR without task descriptions (ablation) | CHEM-BERT · MoleculeNet classification | average score: 83.86 -> 83.11 | Hurt |
| Pretrain on MT-MTR without task descriptions (ablation) | CHEM-BERT · MoleculeNet regression | average score: 0.9243 -> 0.9209 | Helped |
| Pretrain PubMedBERT on MT-MTR without task descriptions (ablation) | PubMedBERT · MoleculeNet classification | average score: 79.87 -> 81.67 | Helped |
| Pretrain PubMedBERT on MT-MTR without task descriptions (ablation) | PubMedBERT · MoleculeNet regression | average score: 1.1949 -> 0.9148 | Helped |
| Append task descriptions after SMILES during pretraining (ablation) | PubMedBERT† · MoleculeNet classification | average score: 79.87 -> 81.08 | Helped |
| Append task descriptions after SMILES during pretraining (ablation) | PubMedBERT† · MoleculeNet regression | average score: 1.1949 -> 0.9131 | Helped |
| Replace task-specific descriptions with irrelevant noise text (ablation) | MolTailor · MoleculeNet classification | average score: 82.92 -> 82.77 | Hurt |
| Replace task-specific descriptions with irrelevant noise text (ablation) | MolTailor · MoleculeNet regression | average score: 0.8348 -> 0.8541 | Hurt |
Surprising findings
- Task descriptions help regression but the pretraining objective hurts classification performance on several datasets.
- Replacing task descriptions with irrelevant text causes only a slight performance drop, suggesting text information contributes less than expected.
Limitations
- Classification performance is inconsistent, with losses on some datasets relative to the backbone.
- Some MoleculeNet molecules overlap with MT-MTR pretraining data, a potential leakage concern (partly addressed by MolTailor*).
- Text prompt ablation shows only a slight drop with noise, so the contribution of text reasoning is unclear.
Experimental setup
Datasets: MT-MTR DrugBank ChEBI MoleculeNet (BBBP, ClinTox, HIV, Tox21, ESOL, FreeSolv, Lipophilicity, QM8)
Benchmarks: MoleculeNet BBBP ClinTox HIV Tox21 ESOL FreeSolv Lipophilicity QM8
Metrics: ROC-AUC RMSE MAE delta AP
Baselines: MACCS-FP RDKit-FP Morgan-FP Grover MolCLR Mole-BERT Uni-Mol BioLinkBERT PubMedBERT ChemBERTa-2 CHEM-BERT KCL
Builds on: PubMedBERT CHEM-BERT ChemBERTa-2 MoleculeNet Optuna
Closest papers by meaning
- Omni-Mol: Multitask Molecular Model for Any-to-any Modalities (NeurIPS 2025 · similarity 0.89)
- Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model (NeurIPS 2025 · similarity 0.89)
- KnowMol: Advancing Molecular Large Language Models with Multi-Level Chemical Knowledge (NeurIPS 2025 · similarity 0.88)
- MolVision: Molecular Property Prediction with Vision Language Models (NeurIPS 2025 · similarity 0.88)
- Domain-Agnostic Molecular Generation with Chemical Feedback (ICLR 2024 · similarity 0.87)
- LICO: Large Language Models for In-Context Molecular Optimization (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.