Research map / Graph learning / Graph self-supervised learning
Position: Graph Foundation Models Are Already Here
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A position paper arguing that graph foundation models are already feasible by framing their key design as finding a transferable "graph vocabulary," grounded in network analysis, expressiveness, and stability principles, and reviewing existing GFMs.
Problem
Building graph foundation models that transfer across diverse graphs, tasks, and domains is hard because graphs lack a shared representation space, unlike text and images; there is no general guidance on achieving positive transfer.
Approach
Proposes a "graph vocabulary" perspective: the basic transferable units encoding graph invariance should be identified and constructed, with principles from network analysis, expressiveness (most-expressive structural representations), and stability guiding vocabulary design, and explains existing GFMs (e.g., ULTRA) as successful vocabulary constructions.
Key results
- No new experiments; the paper argues via review of prior results (e.g., ULTRA zero-shot KGC across datasets, OFA comparable to GNNs on each task).
- Cataloged existing GFMs into primitive, domain-specific, and task-specific categories (e.g., PRODIGY, OneForAll, LLaGA, DiG, MoleBERT, ULTRA, G-Retriever).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ULTRA’s relational vocabulary and double-equivariant relation graph | ULTRA · Knowledge graph completion across datasets | zero-shot performance | Helped |
| Co-train across citation networks, molecule graphs, and knowledge graphs with a unified task formulation | OneForAll (OFA) · Node-, link-, and graph-level tasks | performance | Helped |
| Increase the number of parameters | GAT · Graph regression tasks | performance | Hurt |
| Use larger LLMs as textual feature encoders | LLM feature encoders · Graph tasks using textual features | performance | No clear effect |
| Flatten graph structures into prompts without an additional structure encoder | LLM predictors · Conventional graph tasks | performance | No clear effect |
| Use node-level pretext tasks for generative pre-training | Generative graph pre-training · Graph-level downstream tasks | performance | Hurt |
| Use link prediction directly as a pretext task rather than reformulating node classification as link prediction | Graph pre-training · Node classification | transfer performance | Hurt |
Surprising findings
- Graph-trained LLMs ahead of GFM claims: CiteSeer has over 15% mislabeled data, raising doubts about graph supervision quality.
- Scaling law does not necessarily hold for LLM feature encoders on text understanding tasks, so larger LLMs do not always improve performance.
Limitations
- No original experiments; claims rest on synthesis of prior literature.
- Universal graph vocabulary remains unrealized; current vocabularies are task- or domain-specific.
Experimental setup
Datasets: Ogbn-arxiv Arxiv-year CiteSeer NetworkRepository TU-Dataset OGB MalNet MoleculeNet
Baselines: GCN GAT R-GCN CompGCN
Builds on: ULTRA NBFNet OFA VQGraph GraphGPT
Closest papers by meaning
- GFT: Graph Foundation Model with Transferable Tree Vocabulary (NeurIPS 2024 · similarity 0.91)
- Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights (NeurIPS 2024 · similarity 0.90)
- GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning (NeurIPS 2025 · similarity 0.88)
- GOFA: A Generative One-For-All Model for Joint Graph Language Modeling (ICLR 2025 · similarity 0.87)
- Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement (NeurIPS 2025 · similarity 0.87)
- Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.