Research map / Large language models / Agents, code and math
From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates 1M pseudo molecule-description pairs with ChatGPT via retrieval-based one-shot prompting, then uses them for domain adaptation (pre-training then fine-tuning) or data augmentation to train small T5 models for molecule captioning and text-to-molecule generation.
Problem
Cross-modal molecule discovery (molecule captioning and text-based de novo generation) is limited by the scarcity of parallel molecule-description pairs, which are costly to annotate with chemistry expertise.
Approach
Use LLM-generated "artificially-real" parallel data built from unlabeled PubChem molecules, with retrieval of Tanimoto-similar annotated molecules as one-shot in-context examples, and use the pseudo data as a second pre-training (domain adaptation) stage before fine-tuning on real data.
Key results
- Ada-T5-base (250M) achieves BLEU-4 0.564 on ChEBI-20 captioning, beating Text&Chem T5 (0.542) and MolT5 (0.508) with less pre-training data (1M vs. 33.5M-500M).
- Ada-T5 (77M) reaches 0.449 accuracy on ChEBI-20 molecule generation versus 0.322 for Text&Chem T5 and 0.311 for MolT5.
- Ada-T5 keeps improving as pseudo data grows from 1k to 512k, while Aug-T5 degrades beyond about 4k-32k pseudo samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use pseudo data for domain adaptation, then fine-tune on real data instead of mixing pseudo and real data during fine-tuning | T5 · ChEBI-20, PCdes, and DrugBank-23 molecular captioning and molecule generation | task metrics | Helped |
| Increase pseudo-data volume during domain adaptation (ablation) | Ada-T5 · ChEBI-20 molecular captioning and molecule generation | overall metrics | Helped |
| Increase pseudo-data volume during fine-tuning augmentation (ablation) | Aug-T5 · ChEBI-20 molecular captioning | overall metrics | Hurt |
| Increase model scale from T5_small to T5_base for data augmentation (ablation) | Aug-T5 · Molecular captioning | overall metrics | No clear effect |
| Increase model scale from T5_small to T5_base for domain adaptation (ablation) | Ada-T5 · Molecular captioning | overall metrics | Helped |
| Fine-tune with a mixture of real and pseudo data rather than fine-tuning on real data after pseudo-data domain adaptation | Ada-T5 versus Aug-T5 · Molecule generation | overall task metrics | Hurt |
| Increase pseudo-data volume during fine-tuning augmentation (ablation) | Aug-T5 · ChEBI-20 molecule generation | overall metrics | Mixed |
Surprising findings
- Aug-T5 performance declines as pseudo data grows past a few thousand samples, whereas domain-adaptation Ada-T5 keeps improving.
- Pseudo data generated by ChatGPT lets trained models outperform ChatGPT itself on all metrics, despite the pseudo data coming from ChatGPT.
Limitations
- Aug-T5 is unstable: performance drops with more pseudo data, and pseudo descriptions are only truly pseudo for the generation task since SMILES are real.
- Pseudo data may be unsuitable for direct real-world application because its content does not reflect real observations.
- Generated pseudo data depends on a closed ChatGPT API, so reproducibility and cost depend on a proprietary model.
Experimental setup
Datasets: ChEBI-20 PCdes DrugBank-23 PseudoMD-1M PubChem unlabeled SMILES
Benchmarks: ChEBI-20 PCdes DrugBank-23
Metrics: BLEU-4 ROUGE-2 METEOR Accuracy Validity MACCS FTS RDK FTS Morgan FTS FCD Levenshtein distance
Baselines: T5 MolT5 ChatGPT MolXPT Text&Chem T5
Builds on: MolT5 Text2Mol in-context learning Text&Chem T5
Closest papers by meaning
- Efficient Evolutionary Search Over Chemical Space with Large Language Models (ICLR 2025 · similarity 0.88)
- KnowMol: Advancing Molecular Large Language Models with Multi-Level Chemical Knowledge (NeurIPS 2025 · similarity 0.88)
- Atomas: Hierarchical Adaptive Alignment on Molecule-Text for Unified Molecule Understanding and Generation (ICLR 2025 · similarity 0.87)
- Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages (ICML 2025 · similarity 0.87)
- nach0-pc: Multi-task Language Model with Molecular Point Cloud Encoder (AAAI 2025 · similarity 0.87)
- LICO: Large Language Models for In-Context Molecular Optimization (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.