Research map / Graph learning / Graph self-supervised learning
GFT: Graph Foundation Model with Transferable Tree Vocabulary
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GFT, a graph foundation model that treats computation trees as transferable tokens in a VQ-learned tree vocabulary, pre-trained with tree reconstruction and fine-tuned via computation-tree classification.
Problem
Graph foundation models need a transferable vocabulary across tasks and domains; subgraph-based vocabularies are costly to extract and message-passing GNNs fail to capture some substructures, and graphon theory's assumptions rarely hold on real cross-domain graphs.
Approach
Use computation trees (unfolded message-passing subtrees) as the transferable vocabulary tokens: learn a discrete tree vocabulary by vector quantization during pre-training with feature, semantic, and topology tree-reconstruction losses, then unify node, link, and graph tasks as computation-tree classification with prototype and linear classifiers while keeping the vocabulary fixed.
Key results
- In the pre-training and fine-tuning setting, GFT averages 79.92 across eight datasets, beating the best baseline (GAT, 75.44 average) by over 4 points.
- In few-shot learning on Arxiv 40-way 5-shot, GFT reaches 36.29 versus 25.51 for Prodigy and 24.01 for OFA.
- GFT uses about 7M parameters, compared with 2M for Prodigy, 29M for OFA, and 180M for UniGraph.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace computation trees with subgraphs (ablation) | GFT · Node classification | accuracy: 75.51 -> 74.23 | Hurt |
| Replace computation trees with subgraphs (ablation) | GFT · Link classification | accuracy: 88.99 -> 86.49 | Hurt |
| Replace computation trees with subgraphs (ablation) | GFT · Graph classification | accuracy: 72.21 -> 67.89 | Hurt |
| Replace computation trees with subgraphs (ablation) | GFT · Average | accuracy: 78.06 -> 76.13 | Hurt |
| Combine semantic, feature, and topology reconstruction (ablation) | GFT pre-training · Node-, link-, and graph-level tasks | performance | Helped |
| Add semantic reconstruction (ablation) | GFT pre-training · Node-level and graph-level tasks | performance | Helped |
| Add feature reconstruction (ablation) | GFT pre-training · Link-level tasks | performance | Helped |
| Combine prototype and linear classifiers (ablation) | GFT fine-tuning · Graph-learning tasks | performance | Helped |
| Remove strategies that enhance tree-vocabulary quality (ablation) | GFT · All settings | performance | Hurt |
| Use a tree vocabulary (ablation) | GFT · Link- and graph-level tasks | performance | Helped |
| Increase the number of vocabulary tokens (ablation) | GFT · Graph-learning tasks | performance | No clear effect |
| Pre-train on datasets excluding the target dataset instead of only on the target dataset | GFT · Cross-domain and cross-task transfer | performance | Helped |
Surprising findings
- Euclidean distance for querying tree tokens caused vocabulary collapse, so the authors switched to cosine distance.
- Pre-training on datasets excluding the target outperformed pre-training solely on the target dataset.
Limitations
- Analysis and method focus on message-passing GNNs; graph transformers and expressive GNNs are excluded.
- Message-passing GNNs may lack expressiveness for graph isomorphism.
- Heterophily graphs were not evaluated in the experiments; the performance gains are reported only on homophily and text-attributed graphs.
Experimental setup
Datasets: Cora PubMed Arxiv WikiCS WN18RR FB15K237 HIV PCBA ChEMBL
Benchmarks: Cora PubMed Wiki-CS Arxiv WN18RR FB15K237 HIV PCBA
Metrics: accuracy ROC-AUC hits@1
Baselines: GCN GAT GIN DGI BGRL GraphMAE GIANT Prodigy OFA
Builds on: VQ-VAE VQGraph OFA Prodigy GraphMAE
Closest papers by meaning
- Position: Graph Foundation Models Are Already Here (ICML 2024 · similarity 0.91)
- Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees (ICML 2025 · similarity 0.89)
- Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights (NeurIPS 2024 · similarity 0.88)
- Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement (NeurIPS 2025 · similarity 0.88)
- GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning (NeurIPS 2025 · similarity 0.87)
- How Much Can Transfer? BRIDGE: Bounded Multi-Domain Graph Foundation Model with Generalization Guarantees (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.