Research map / Graph learning / Graph self-supervised learning
Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes task-trees as unified learning instances for node, edge, and graph tasks, with theory on stability, transferability and generalization, and builds GIT, a GraphSAGE-based graph foundation model pretrained on task-trees via a reconstruction objective.
Problem
Cross-task generalization in graph foundation models is hard because node-, edge-, and graph-level tasks operate on different learning units, and subgraph-based instances are costly and hard for message-passing GNNs to learn.
Approach
Treat the computation tree rooted at a virtual task node connected to all task-relevant nodes as the unified learning instance; encode it by appending virtual nodes to the original graph, pretrain with a two-view reconstruction objective plus KL alignment, then fine-tune, use in-context prototypes, zero-shot LLM label embeddings, or specialize via instruction tuning.
Key results
- GIT-S reaches 36.32 average zero-shot score versus 26.14 for OFA across five domains.
- GIT-G achieves 75.37 average fine-tuning score, above OFA's 73.08 and GraphMAE's 72.79.
- In molecular graphs GIT-S scores 62.83, second only to GIMLET (64.15) among domain experts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| specialize GIT-G using domain data | GIT-G · Held-out Avg. (0-shot) | score: 28.56 -> 35.19 | Helped |
| specialize GIT-G using domain data | GIT-G · Held-out Avg. (3-shot) | score: 56.09 -> 58.69 | Helped |
| specialize GIT-G using domain data | GIT-G · Held-out Avg. (Finetune) | score: 73.84 -> 74.19 | Helped |
| use task-trees instead of subgraphs (ablation) | GIT · Academia (Finetune) | score: 73.48 -> 75.82 | Helped |
| use task-trees instead of subgraphs (ablation) | GIT · KG (Finetune) | score: 73.59 -> 75.73 | Helped |
| use task-trees instead of subgraphs (ablation) | GIT · Molecule (Finetune) | score: 72.67 -> 75.73 | Helped |
| apply SVD and pretraining (ablation) | GIT-G · Cora (link prediction) | AUC: 94.27 -> 95.70 | Helped |
| add domain regularizer with λ = 10 (ablation) | GIT · Eight datasets (0-shot) | score: 20.39 -> 29.60 | Helped |
| add domain regularizer with λ = 10 (ablation) | GIT · Eight datasets (3-shot) | score: 53.10 -> 60.21 | Helped |
| add domain regularizer with λ = 10 (ablation) | GIT · Eight datasets (Finetune) | score: 74.78 -> 75.37 | Helped |
| increase domain-regularizer weight from λ = 1 to λ = 10 (ablation) | GIT · Eight datasets (Finetune) | score: 75.41 -> 75.37 | No clear effect |
| specialize the general model on Arxiv (ablation) | GIT · Academic networks (0-shot) | score: 14.88 -> 23.45 | Helped |
Surprising findings
- Scaling pretraining data did not yield a clear performance trend, despite scaling hidden dimensions helping.
- GraphMAE and OFA degrade when moving from expert or base models to a general pretrained model, while GIT stays stable.
Limitations
- Converting graph neighborhoods into trees loses information due to message-passing GNN expressiveness limits (1-WL).
- Scaling pretraining data gave no clear gains, suggesting limited encoder capacity or dataset misalignment.
- Relies on text-attributed graphs for feature alignment; non-text graphs need an extra SVD module.
Experimental setup
Datasets: Arxiv Products WN18RR FB15K237 Chemblpre PCBA Cora PubMed Citeseer IMDB-B
Benchmarks: Arxiv Products FB15K237 PCBA Cora
Metrics: accuracy AUC
Baselines: GCN GAT GIN BGRL GraphMAE OFA GraphPrompt+ All in One OpenGraph AnyGraph
Builds on: GraphSAGE GraphMAE OFA GFT BGRL
Closest papers by meaning
- A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning (NeurIPS 2026 · similarity 0.90)
- UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains (NeurIPS 2025 · similarity 0.90)
- GFT: Graph Foundation Model with Transferable Tree Vocabulary (NeurIPS 2024 · similarity 0.89)
- GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning (NeurIPS 2025 · similarity 0.89)
- Generative Graph Pattern Machine (NeurIPS 2025 · similarity 0.88)
- One For All: Towards Training One Graph Model For All Classification Tasks (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.