Research map / Label-efficient and robust learning / Knowledge distillation
From Coarse to Fine: Enable Comprehensive Graph Self-supervised Learning with Multi-granular Semantic Ensemble
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MGSE, a plug-and-play self-supervised knowledge distillation framework that trains multiple student graph encoders against a frozen graph SSL teacher at different prototype granularities and averages them for downstream use.
Problem
Existing graph SSL frameworks capture either high-level abstract or fine-grained features but not both, which limits generalization across downstream graph tasks, especially under distribution shift.
Approach
Distill a frozen pre-trained graph SSL teacher into K randomly initialized students, each aligned to the teacher via prototype-based soft assignments with a different number of prototypes (granularity), with sharpening and ME-MAX regularization, then ensemble the fine-tuned students by averaging predictions.
Key results
- MGSE applied to six graph SSL teachers improves average ROC-AUC on the eight chemistry datasets by roughly 0.7 to 3.1 points per teacher (e.g., GraphCL avg +1.21 across datasets).
- Largest reported gain is +7.02 ROC-AUC (9.2%) for GraphLoG on ClinTox.
- Averaging student predictions outperforms stacking and attentive ensembles on four datasets; ablations show removing sharpening or ME-MAX reduces gains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MGSE to GraphCL | GraphCL · BBBP | ROC-AUC: 69.68±0.67 -> 72.26±0.65 | Helped |
| Add MGSE to GraphCL | GraphCL · Tox21 | ROC-AUC: 73.87±0.66 -> 75.89±0.33 | Helped |
| Add MGSE to GraphLoG | GraphLoG · ClinTox | ROC-AUC: 76.7±3.3 -> 83.72±2.02 | Helped |
| Add MGSE to GraphLoG | GraphLoG · BACE | ROC-AUC: 83.5±1.2 -> 83.18±1.24 | Hurt |
| Add MGSE to GraphMAE | GraphMAE · BBBP | ROC-AUC: 72.0±0.6 -> 71.62±0.51 | Hurt |
| Add MGSE across the reported graph SSL methods | Graph SSL frameworks · Tox21 | ROC-AUC: +1.21 (1.6%) | Helped |
| Add target-distribution sharpening (ablation) | GraphCL+MGSE · BBBP | ROC-AUC: 70.91±0.89 -> 72.26±0.65 | Helped |
| Add ME-MAX regularization (ablation) | GraphCL+MGSE · SIDER | ROC-AUC: 60.08±0.82 -> 61.44±0.68 | Helped |
| Use multiple granularities rather than only granularity #1 (DK = 2) (ablation) | GraphCL+MGSE · ToxCast | ROC-AUC: 61.75±0.71 -> 64.57±0.34 | Helped |
| Replace multi-granularity distillation with multi-teacher dropout ensemble (ablation) | GraphCL · Tox21 | ROC-AUC: 73.85±0.74 -> 75.89±0.33 | Hurt |
| Use more than one student model (ablation) | MGSE · ClinTox | ROC-AUC | Helped |
| Use more than one student model (ablation) | MGSE · BACE | ROC-AUC | No clear effect |
Surprising findings
- Pre-trained GraphLoG prototypes reused for distillation (Old Proto) performed worse, with distilled students not matching the teacher, so new prototypes were needed.
- Gains are smaller for teachers using strong structural augmentation (AD-GCL, RGCL) and for some generative teachers (GraphMAE slightly drops on BBBP).
Limitations
- Extra computation cost from training K student models scales linearly.
- Evaluated mainly on chemistry and biology graph-level tasks with limited node-level experiments in the main text.
Experimental setup
Datasets: ZINC15 PPI-306K MoleculeNet (BBBP, Tox21, ToxCast, SIDER, ClinTox, HIV, BACE, MUV) PPI Cora Citeseer Pubmed ogbn-arxiv
Benchmarks: MoleculeNet PPI Cora Citeseer Pubmed ogbn-arxiv
Metrics: ROC-AUC accuracy
Baselines: GraphCL JOAO AD-GCL GraphLoG RGCL GRACE GraphMAE CCA-SSG No pre-train
Builds on: GraphLoG self-distillation (Allen-Zhu & Li, 2023) SwAV-style prototypes / ME-MAX DINO-style self-distillation (Caron et al., 2021) CCL / Fang et al. 2020 self-supervised distillation
Closest papers by meaning
- Core Knowledge Learning Framework for Graph (AAAI 2025 · similarity 0.87)
- SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs (NeurIPS 2025 · similarity 0.87)
- Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement (NeurIPS 2025 · similarity 0.86)
- Uncovering the Redundancy in Graph Self-supervised Learning Models (NeurIPS 2024 · similarity 0.86)
- Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs (NeurIPS 2025 · similarity 0.86)
- Every Node Is Different: Dynamically Fusing Self-Supervised Tasks for Attributed Graph Clustering (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.