Research map / Graph learning / Graph self-supervised learning
Exploring Correlations of Self-Supervised Tasks for Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measures correlations between eight graph self-supervised tasks via transfer losses, shows existing multi-task methods fail to beat single tasks, and proposes GraphTCM, a learned correlation model used to train a linear mixing of task representations that improves node classification and link prediction.
Problem
Graph self-supervised tasks are used in isolation or combined by multi-task losses, but their inter-task relationships are unexplored and it is unclear whether combining them yields representations that generalize across self-supervised and downstream tasks.
Approach
Defines a correlation value Cor(t1,t2) as the ratio of the best linear-probe loss on task t2 using representations from t1 to that using representations from t2, and trains a GraphTCM (query-key exponential model over readout representations) to predict these correlations; then uses it as an objective to learn a linear mixture of base task representations that minimizes correlation values across t
Key results
- GraphTCM-enhanced representations achieve the best node classification accuracy on all six datasets, e.g., 81.50% on Cora versus 77.28% for the best single base task (SubgCon).
- On link prediction, GraphTCM reaches 95.52% (Cora), 96.14% (CiteSeer) and 97.13% (PubMed) ROC-AUC, outperforming all baselines.
- GraphTCM models correlation values with 6.04% training and 8.97% validation error, and 13.56% average error on three unseen tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GraphTCM-enhanced representations instead of the strongest baseline representations | GraphTCM-enhanced representations · Node classification across six datasets | accuracy: 1.7% average improvement | Helped |
| Use GraphTCM-enhanced representations instead of the strongest baseline representations | GraphTCM-enhanced representations · Link prediction on Cora, CiteSeer and PubMed | ROC-AUC: 2.2% average improvement | Helped |
| Add the learnable W_r parameter to GraphTCM (ablation) | GraphTCM · Cora, CiteSeer, PubMed correlation modeling | average validating error (%): 105.36 -> 8.97 | Helped |
| Add the learnable W_t parameter to GraphTCM (ablation) | GraphTCM · Cora, CiteSeer, PubMed correlation modeling | average validating error (%): 128.30 -> 8.97 | Helped |
| Add the exponential function to correlation computation (ablation) | GraphTCM · Cora, CiteSeer, PubMed correlation modeling | average validating error (%): 51.05 -> 8.97 | Helped |
Surprising findings
- Existing multi-task methods (Multi-loss, AutoSSL, ParetoGNN) fail to outperform the best single self-supervised task on representation capability and often degrade downstream performance.
- Task difficulty and representation capability are not consistently correlated, and task correlations vary across datasets.
Limitations
- Correlation values are computed per dataset, so GraphTCM must be retrained for each dataset.
- Only eight base tasks from four categories were studied, with two representative methods per category.
Experimental setup
Datasets: Cora CiteSeer PubMed ogbn-arxiv Amazon-Computers Amazon-Photo
Benchmarks: Cora CiteSeer PubMed ogbn-arxiv Amazon-Computers Amazon-Photo
Metrics: accuracy ROC-AUC ARL ATD
Baselines: Multi-loss AutoSSL ParetoGNN Representation addition Representation concat GraphComp AttributeMask GAE EdgeMask NodeProp DisCluster DGI
Builds on: DGI GAE AutoSSL ParetoGNN GraphMAE
Closest papers by meaning
- Every Node Is Different: Dynamically Fusing Self-Supervised Tasks for Attributed Graph Clustering (AAAI 2024 · similarity 0.88)
- Uncovering the Redundancy in Graph Self-supervised Learning Models (NeurIPS 2024 · similarity 0.86)
- One For All: Towards Training One Graph Model For All Classification Tasks (ICLR 2024 · similarity 0.86)
- Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery (AAAI 2024 · similarity 0.85)
- Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees (ICML 2025 · similarity 0.85)
- Revisiting Self-Supervised Heterogeneous Graph Learning from Spectral Clustering Perspective (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.