Research map / Graph learning / GNN architectures
Pre-Training Graph Neural Networks on Molecules by Using Subgraph-Conditioned Graph Information Bottleneck
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S-CGIB, a self-supervised graph information bottleneck pre-training method for molecular GNNs that learns graph cores conditioned on ego-network functional-group candidates, without labels or chemistry priors.
Problem
Pre-training GNNs on unlabeled molecules without human annotations or prior knowledge of functional groups, while producing well-separated graph-level representations and discovering functional groups automatically.
Approach
Extends the graph information bottleneck to a self-supervised, subgraph-conditioned form (S-CGIB): a noise-injected graph core is trained to reconstruct the input graph's adjacency conditioned on ego-network functional-group candidates, which are weighted by attention between the core and the candidates; a contrastive term and an unsupervised node-feature domain adaptation step are added.
Key results
- S-CGIB achieved the best ROC-AUC on 10 of 11 graph classification datasets, e.g., BBBP 88.75 vs. 87.15 for GROVE and Tox21 80.94 vs. 78.29 for MoAMa.
- On Peptides-func, S-CGIB reached AP 0.694 versus 0.618 for GraphFP, reported as a 12.3% improvement.
- On the interpretability benchmarks, S-CGIB obtained Fidelity- 0.008 on Mutagenicity, versus 0.150 for MGSSL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add domain adaptation (ablation) | S-CGIB · BBBP | ROC-AUC: 86.71$\pm$0.74 -> 88.75$\pm$0.49 | Helped |
| add domain adaptation (ablation) | S-CGIB · FreeSolv | RMSE: 1.832$\pm$0.095 -> 1.648$\pm$0.074 | Helped |
| add graph core (ablation) | S-CGIB with attention coefficients · BBBP | ROC-AUC: 83.41$\pm$0.71 -> 88.75$\pm$0.49 | Helped |
| add attention coefficients for significant subgraphs (ablation) | S-CGIB with graph core · BBBP | ROC-AUC: 86.26$\pm$0.63 -> 88.75$\pm$0.49 | Helped |
| add fully connected decoder layers (ablation) | S-CGIB with adjacency reconstruction · BBBP | ROC-AUC: 88.75$\pm$0.49 -> 88.06$\pm$0.37 | Hurt |
| replace adjacency reconstruction with k-step transition reconstruction (ablation) | S-CGIB · BBBP | ROC-AUC: 88.75$\pm$0.49 -> 86.14$\pm$0.31 | Hurt |
| use a more expressive GIN encoder instead of GCN, GraphSage, or GAT (ablation) | S-CGIB · BBBP | ROC-AUC: GCN 76.53$\pm$0.48; GraphSage 83.04$\pm$0.38; GAT 53.31$\pm$0.61; GIN 88.75$\pm$0.49 | Helped |
| increase subgraph size beyond k=3 (ablation) | S-CGIB · molecular property prediction datasets | performance | Hurt |
| add a positive weighted I(G; S) term (ablation) | S-CGIB pre-training objective · molecular property prediction datasets | performance | Hurt |
| add pre-training and domain adaptation (ablation) | S-CGIB · downstream tasks | convergence speed | Helped |
Surprising findings
- Adding the I(G;S) term to the objective severely degraded performance, so the authors minimized only the conditional reconstruction term.
- Using only the attention-based subgraph interaction (without the graph core) was reported as slightly more beneficial than using only the graph core in the ablation on certain datasets.
Limitations
- Pre-training uses only 300k molecules and small-scale molecular datasets, limiting scale.
- Reliance on fixed hyperparameters such as the subgraph size k and GIN depth, with performance sensitive to k (best for k<=3).
- Pre-training can hurt weaker encoders (GCN, GraphSAGE, GAT), so benefits depend on an expressive GIN encoder.
Experimental setup
Datasets: PCQ4Mv2 QM9 mol-PCBA mol-HIV BACE BBBP Tox21 ToxCast SIDER ClinTox MUV ESOL
Benchmarks: MoleculeNet TUDataset LRGB BENZENE Alkane Carbonyl Fluoride Carbonyl
Metrics: ROC-AUC accuracy RMSE MAE AP Fidelity+ Fidelity- JSD
Baselines: ContextPred AttrMasking EdgePred Infomax JOAO JOAOv2 GraphCL GraphLoG GraphFP MICRO-Graph MGSSL GROVER
Builds on: GIB CGIB VGIB GIN GraphCL GraphFP
Closest papers by meaning
- On the Scalability of GNNs for Molecular Graphs (NeurIPS 2024 · similarity 0.89)
- Neural Atoms: Propagating Long-range Interaction in Molecular Graphs through Efficient Communication Channel (ICLR 2024 · similarity 0.87)
- MAGE: Model-Level Graph Neural Networks Explanations via Motif-based Graph Generation (ICLR 2025 · similarity 0.87)
- Robust Heterogeneous Graph Classification for Molecular Property Prediction with Information Bottleneck (AAAI 2025 · similarity 0.87)
- Extracting Training Data from Molecular Pre-trained Models (NeurIPS 2024 · similarity 0.87)
- MORE: Molecule Pretraining with Multi-Level Pretext Task (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.