Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ToyMix, LargeMix, and UltraLarge multi-task, multi-level molecular datasets (~100M molecules, 13B labels) and the Graphium library, with GCN/GIN/GINE multi-task baselines showing gains on small bio datasets from quantum co-training.
Problem
Molecular machine learning lacks large, labeled, multi-task datasets combining quantum and biological labels, which hinders building foundation models for molecules.
Approach
Curates seven datasets grouped into ToyMix, LargeMix and UltraLarge mixes spanning quantum (DFT/PM6), bioassay and transcriptomic labels with sparse graph- and node-level tasks, plus a modular library (Graphium) for multi-task, multi-level training with missing-label handling and muP tuning.
Key results
- On ToyMix, GINE outperforms GIN, which outperforms GCN across settings; Zinc12k and Tox21 gain clearly from multi-dataset training.
- On LargeMix, the small L1000 VCAP and MCF7 datasets benefit from multi-dataset training, while PCQM4M and PCBA_1328 do not.
- On UltraLarge (5% subset), multi-task training shows no performance advantage for graph- or node-level tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with the combined ToyMix datasets instead of training on a single dataset | GNN baselines · ZINC12k | performance | Helped |
| Train with the combined ToyMix datasets instead of training on a single dataset | GNN baselines · Tox21 | performance | Helped |
| Train with the combined ToyMix datasets instead of training on a single dataset | GNN baselines · QM9 | performance | Hurt |
| Train with the combined LargeMix datasets instead of training on a single dataset | GNN baselines · L1000 VCAP | performance | Helped |
| Train with the combined LargeMix datasets instead of training on a single dataset | GNN baselines · L1000 MCF7 | performance | Helped |
| Train with the combined LargeMix datasets instead of training on a single dataset | GNN baselines · PCQM4M_G25_N4 | performance | No clear effect |
| Train with the combined LargeMix datasets instead of training on a single dataset | GNN baselines · PCBA_1328 | performance | No clear effect |
| Use multi-task training instead of single-dataset training | GNN baselines · PM6_83M | performance | No clear effect |
Surprising findings
- On UltraLarge, multi-task training gave no performance advantage over single-dataset training.
- Multi-task training slightly reduced QM9 performance in ToyMix.
Limitations
- No edge- or node-pair-level tasks in the current datasets.
- Baselines limited to three GNNs; no graph transformers or extensive positional encoding ablations.
- UltraLarge baselines trained only on a 5% subset due to resource constraints.
Experimental setup
Datasets: QM9 ZINC12k Tox21 PCQM4M_G25_N4 PCBA_1328 L1000_VCAP L1000_MCF7 PM6_83M
Benchmarks: ToyMix LargeMix UltraLarge
Metrics: MAE Pearson correlation R^2 AUROC average precision
Baselines: GCN GIN GINE
Builds on: PubChemQC OGB-PCQM4M OGB-PCBA muP GraphGym
Closest papers by meaning
- MORE: Molecule Pretraining with Multi-Level Pretext Task (AAAI 2025 · similarity 0.89)
- Bridging Molecular Graphs and Large Language Models (AAAI 2025 · similarity 0.88)
- Monroe: A Molecular Foundation Model for In-context Probabilistic Inference (NeurIPS 2026 · similarity 0.88)
- Molecular Data Programming: Towards Molecule Pseudo-labeling with Systematic Weak Supervision (CVPR 2024 · similarity 0.88)
- On the Scalability of GNNs for Molecular Graphs (NeurIPS 2024 · similarity 0.87)
- Omni-Mol: Multitask Molecular Model for Any-to-any Modalities (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.