Research map / Label-efficient and robust learning / Recognition and long-tail
Measuring Task Similarity and Its Implication in Fine-Tuning Graph Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a task consistency measure over a pair-wise label space to predict when a graph pre-trained model benefits a downstream task, and Bridge-Tune, a fine-tuning strategy that adds an iterative pre-trained model refinement step maximizing representation consistency.
Problem
Pre-trained GNNs are often treated as universally transferable, but fine-tuning frequently yields limited or negative transfer depending on the gap between pre-training and downstream tasks; existing task similarity measures rely on labels and do not apply to unlabeled graph pre-training.
Approach
Map pre-training and downstream tasks into a shared pair-wise label space (whether two nodes share a label or are linked), define task consistency as the agreement rate between these pair labels, and use a representation consistency measure to iteratively refine the pre-trained encoder between fine-tuning steps (Bridge-Tune).
Key results
- On GCC node classification, Bridge-Tune reaches 68.99 Micro F1 on US-Airport vs 65.82 for the best baseline GTOT, averaging +4.68% over the best baseline across tasks.
- Bridge-Tune improves link prediction on US-Airport with GCC from 90.76 (GTOT) to 94.97 Micro F1 (+4.64%).
- Pre-trained GCC fine-tuned by naive methods suffers negative transfer on 45.5% of downstream tasks, and task consistency correlates positively with performance improvement.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bridge-Tune instead of GTOT | GCC · US-Airport node classification | Micro F1: 4.82% ↑ | Helped |
| Use Bridge-Tune instead of GTOT | GCC · US-Airport link prediction | Micro F1: 4.64% ↑ | Helped |
| Use Bridge-Tune instead of GTOT | GraphCL · US-Airport node classification | Micro F1: 1.54% ↑ | Helped |
| Use Bridge-Tune instead of GTOT | GraphCL · US-Airport link prediction | Micro F1: 2.98% ↑ | Helped |
| Add the time-varying strategy (ablation) | Bridge-Tune on GCC · US-Airport node classification | Micro F1: 67.25% -> 68.99% | Helped |
| Use unlabeled-node predictions when estimating refinement loss (ablation) | Bridge-Tune on GCC · US-Airport node classification | Micro F1: 68.32% -> 68.99% | Helped |
Surprising findings
- Many advanced fine-tuning baselines (e.g., L2_SP, DELTA) perform worse than naive fine-tuning.
- Prompt-tuning methods (GPPT, GraphPrompt) underperform, possibly because they assume pre-training and fine-tuning use the same dataset.
Limitations
- Refinement requires downstream labels or confident predictions, and relies on an iterative procedure that adds cost.
- Theoretical guarantees rely on stated assumptions; the formal analysis is in the appendix.
- Experiments use only GCC, GraphCL, EdgePred, and ContextPred on node-level tasks in the reported main results.
Experimental setup
Datasets: US-Airport Brazil-Airport Europe-Airport H-index Wisconsin Texas Cora Cornell DD242 DD68 DD687 ogbn-arxiv
Benchmarks: US-Airport Brazil-Airport Europe-Airport H-index Wisconsin Texas Cora Cornell DD242 DD68 DD687 ogbn-arxiv
Metrics: Micro F1 relative improvement over training from scratch
Baselines: Fine-tune Freeze Rand L2_penalty L2_SP L2_SP_Fisher DELTA GTOT SupCon L2P GPPT GraphPrompt
Builds on: GCC GTOT InfoNCE GraphCL
Closest papers by meaning
- Edge Prompt Tuning for Graph Neural Networks (ICLR 2025 · similarity 0.88)
- HeterGP: Bridging Heterogeneity in Graph Neural Networks with Multi-View Prompting (AAAI 2025 · similarity 0.88)
- Fine-Tuning Graph Neural Networks by Preserving Graph Generative Patterns (AAAI 2024 · similarity 0.88)
- HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters (ICLR 2025 · similarity 0.87)
- Decoupling Weighing and Selecting for Integrating Multiple Graph Pre-training Tasks (ICLR 2024 · similarity 0.87)
- Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.