Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
MORE: Molecule Pretraining with Multi-Level Pretext Task
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MORE, a GIN-based molecular graph pretraining method combining four pretext tasks (node masking, MACCS subgraph keys, RDKit descriptors, 3D inter-node distances) pretrained on ZINC15 and evaluated on seven MoleculeNet classification tasks.
Problem
Existing molecular graph pretraining tasks capture only single or partial molecular properties (local or global, 2D or 3D, low- or high-level), so learned representations are not comprehensive or generalizable.
Approach
Jointly pretrain one GIN encoder with four decoders, each for a different pretext task at node, subgraph (MACCS keys), graph (RDKit molecular descriptors), and 3D (inter-atomic distances from conformers) levels, to cover multiple views of a molecule under identical architecture.
Key results
- Linear probing average ROC-AUC of 68.24 for MORE versus 63.33 for the second-best method (Infomax) across seven datasets.
- Full fine-tuning average ROC-AUC of 73.40 for MORE, the highest among all compared methods (next best 70.96 for GraphMAE).
- MORE shows the smallest linear-probing-to-fine-tuning gap and the most minimal, stable encoder parameter changes after fine-tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MORE's four-level pretraining rather than the strongest baseline | GIN molecular encoder · MoleculeNet, average across seven datasets, linear probing | ROC-AUC: 63.33 -> 68.24 | Helped |
| Use MORE's four-level pretraining rather than the strongest baseline | GIN molecular encoder · MoleculeNet, average across seven datasets, full fine-tuning | ROC-AUC: 70.96 -> 73.40 | Helped |
| Remove the node-level pretext task (ablation) | MORE · MoleculeNet linear probing | ROC-AUC | Mixed |
| Add the node-level pretext task (ablation) | MORE · BACE linear probing | ROC-AUC | Helped |
| Remove the subgraph-level pretext task (ablation) | MORE · MoleculeNet linear probing | ROC-AUC | Mixed |
| Add the subgraph-level pretext task (ablation) | MORE · MoleculeNet, average across datasets, linear probing | ROC-AUC | Helped |
| Remove the graph-level pretext task (ablation) | MORE · MoleculeNet linear probing | ROC-AUC | Mixed |
| Add the graph-level pretext task (ablation) | MORE · MoleculeNet, average across datasets, linear probing | ROC-AUC | Helped |
| Remove the 3D-level pretext task (ablation) | MORE · MoleculeNet linear probing | ROC-AUC | Mixed |
| Add the 3D-level pretext task (ablation) | MORE · ClinTox linear probing | ROC-AUC | Helped |
| Use only the node-level task rather than all four tasks (ablation) | MORE · MoleculeNet linear probing | ROC-AUC | Mixed |
| Use only the subgraph-level task rather than all four tasks (ablation) | MORE · ToxCast, SIDER, HIV, and BACE linear probing | ROC-AUC | Helped |
Surprising findings
- The graph-level task based on RDKit molecular descriptors, rarely used as a prediction target, gave the best single-task average linear-probing performance among the individual tasks.
Limitations
- Experiments are limited to GNN (GIN) encoders on graph classification; the authors note extension to other architectures is untested.
- Loss weights were tuned on validation sets and downstream fine-tuning used default settings without tuning, so comparisons depend on those choices.
- Pretraining uses 2M ZINC15 molecules with conformers, and scalability is only shown up to 200M-scale subsets described in the text as 10M-200M, with performance trends stated qualitatively.
Experimental setup
Datasets: ZINC15 BBBP Tox21 ToxCast SIDER ClinTox HIV BACE
Benchmarks: BBBP Tox21 ToxCast SIDER ClinTox HIV BACE
Metrics: ROC-AUC performance gap between linear probing and fine-tuning encoder parameter change
Baselines: Infomax EdgePred AttrMasking ContextPred GraphCL GraphLoG GraphMAE GraphMVP
Builds on: GraphMAE GraphMVP GraphCL GraphLoG Infomax
Closest papers by meaning
- Exploring Molecular Pretraining Model at Scale (NeurIPS 2024 · similarity 0.89)
- Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets (ICLR 2024 · similarity 0.89)
- Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains (ICML 2024 · similarity 0.88)
- UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning (ICML 2024 · similarity 0.88)
- Foundation Molecular Grammar: Multi-Modal Foundation Models Induce Interpretable Molecular Graph Languages (ICML 2025 · similarity 0.87)
- From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.