Research map / Graph learning / Knowledge and temporal graphs
Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCR, a graph foundation model pretrained on three KGs (WN18RR, FB15k-237, CodexM) via inductive KG reasoning with semantic-conditioned message passing, transferred zero-shot to node-, link- and graph-level tasks on general graphs.
Problem
Graph foundation models often need fine-tuning to transfer to new graphs; CMP-based KG reasoning models cannot effectively use node semantic features ("semantic isolation"), and task formats differ across link, node and graph level tasks.
Approach
Reformulate node and graph classification as KG link prediction on a task-specific KG, and add semantic-conditioned message passing (SCMP) that injects top-k semantic neighbors via a trainable initialization and a frozen, non-parametric semantic CMP branch merged by an MLP.
Key results
- On 24 inductive KG reasoning datasets, SCR(3g) reaches total average MRR 0.451 and Hits@10 0.633, matching the best ULTRA(4g) MRR and beating ULTRA(3g) (0.433) and the supervised SOTA (0.366).
- On node classification, SCR(3g) achieves 81.80 on Cora, 71.33 on Citeseer and 82.93 on Pubmed, above ULTRA(3g) and graph prompt baselines, but lags GraphAny on heterophilic graphs.
- On graph classification, SCR(3g) reaches average rank 1.75 across 8 datasets, outperforming ULTRA(3g) (rank 8.50) and being competitive with fully trained GIN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add semantic-injected entity initialization (INIT2) (ablation) | SCR · Inductive KG reasoning, node classification, and graph classification | performance | Helped |
| Add semantic-augmented relation graph (SARG) (ablation) | SCR · Inductive KG reasoning, node classification, and graph classification | performance | Helped |
| Add non-parametric semantic representation (NPSR) (ablation) | SCR · Graph tasks, especially inductive KG reasoning | performance | Helped |
| Use relation-specific embeddings instead of identical relation embeddings (ablation) | SCR · Node- and graph-level classification | performance | Helped |
| Reduce the labeled node/graph-label edges (ablation) | SCR · Node and graph classification | accuracy | Hurt |
| Use semantic features from different sentence encoders (MPNet, MiniLM, or DistilBERT) | SCR · KG inductive reasoning | MRR: 0.451 -> 0.451 | No clear effect |
| Use all-ones features instead of the default semantic features | SCR · KG inductive reasoning, Total AVG | MRR: 0.451 -> 0.445 | Hurt |
| Use ontology features instead of the default semantic features | SCR · KG inductive reasoning, Total AVG | MRR: 0.451 -> 0.440 | Hurt |
| Use more KNN semantic neighbors | ULTRA · KG inductive reasoning | reasoning performance | Hurt |
Surprising findings
- Naively injecting BERT features into the CMP initialization degrades performance, revealing the semantic isolation issue.
- Increasing k in a KNN semantic graph reduces ULTRA's reasoning performance.
Limitations
- Focuses only on node semantics; edge semantics not supported in the current design.
- Lags GraphAny on heterophilic node classification, which uses label-based analytical tuning.
- Evaluated only on textual-feature KG datasets; other KGs without accessible text are left for future work.
Experimental setup
Datasets: WN18RR FB15k-237 CodexMedium GraIL inductive datasets InGram inductive datasets Cora Citeseer Pubmed Wisconsin Texas Actor IMDB-B
Benchmarks: GraIL inductive datasets (IndE) InGram inductive datasets (IndER) Cora Citeseer Pubmed Wisconsin Texas Actor IMDB-B COLLAB PROTEINS MUTAG
Metrics: MRR Hits@10 accuracy average rank
Baselines: ULTRA ProLINK OpenGraph AnyGraph GraphAny GPPT All-in-one GPrompt GPF GPF-plus GIN GCN
Builds on: ULTRA NBFNet GraIL SGC
Closest papers by meaning
- Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models (ICML 2025 · similarity 0.91)
- UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains (NeurIPS 2025 · similarity 0.90)
- Towards Foundation Models for Knowledge Graph Reasoning (ICLR 2024 · similarity 0.90)
- Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.90)
- Stability and Generalization Capability of Subgraph Reasoning Models for Inductive Knowledge Graph Completion (ICML 2025 · similarity 0.89)
- KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.