Research map / Graph learning / Knowledge and temporal graphs
Towards Foundation Models for Knowledge Graph Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ULTRA, a graph-of-relations based conditional GNN that yields relation representations invariant to vocabulary, enabling a single model pre-trained on three KGs to perform zero-shot and fine-tuned link prediction on 57 KGs.
Problem
Knowledge graph reasoning models typically cannot transfer across KGs with different entity and relation vocabularies; the paper seeks transferable representations enabling fully inductive (new entities and relations) link prediction as a foundation model for KGs.
Approach
Lift a KG to a graph of relations whose four fundamental interactions (h2h, h2t, t2h, t2t) are vocabulary-independent, then run a conditional GNN with an all-ones indicator on the query relation to produce relative relation representations without any input features, feeding them into an NBFNet-style entity-level link predictor.
Key results
- Zero-shot ULTRA averages 0.395 MRR over 40 graphs vs 0.344 for supervised SOTA; on inductive datasets 0.435 vs 0.342 MRR.
- Fine-tuned ULTRA reaches 0.422 average MRR across 40 graphs, and 0.443 on inductive (e)+(e,r) graphs.
- Ablation: removing the four fundamental interaction types and using an unconditional GNN drops zero-shot total average MRR from 0.366 to 0.192 (random init) on 54 graphs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use pre-trained ULTRA for zero-shot inference instead of dataset-specific supervised models | ULTRA · Total average, 40 graphs | MRR: 0.344 -> 0.395 | Helped |
| Use pre-trained ULTRA for zero-shot inference instead of dataset-specific supervised models | ULTRA · Transductive, 13 graphs | MRR: 0.348 -> 0.312 | Hurt |
| Fine-tune the pre-trained model on each target dataset | ULTRA · Total average, 40 graphs | MRR: 0.395 -> 0.422 | Helped |
| Fine-tune the pre-trained model on each target dataset | ULTRA · Pretraining, 3 graphs | MRR: 0.439 -> 0.407 | Hurt |
| Fine-tune the pre-trained model on each target dataset | ULTRA · Inductive (e,r), 23 graphs | MRR: 0.345 -> 0.397 | Helped |
| Remove edge types from the relation graph (ablation) | ULTRA · Total average, 54 graphs | MRR: 0.366 -> 0.316 | Hurt |
| Remove relation edge types and use an unconditional GNN with all-ones node initialization (ablation) | ULTRA · Total average, 54 graphs | MRR: 0.366 -> 0.199 | Hurt |
| Remove relation edge types and use an unconditional GNN with random node initialization (ablation) | ULTRA · Total average, 54 graphs | MRR: 0.366 -> 0.192 | Hurt |
| Increase the number of graphs in the pre-training mixture beyond three (ablation) | ULTRA · Inductive datasets, 41 graphs | MRR | No clear effect |
Surprising findings
- Pre-training on more graphs does not consistently improve zero-shot performance; performance saturates beyond three graphs in the mixture.
- Scaling the model beyond about 200k parameters did not show significant improvement in preliminary experiments.
Limitations
- Pre-training on more graphs does not reliably help, and the small model size (177k parameters) may limit capacity.
- Zero-shot performance lags supervised baselines on larger transductive graphs such as ConceptNet100k and AristoV4.
- Only structural information is used; no text or entity features are incorporated.
Experimental setup
Datasets: FB15k-237 WN18RR CoDEx-Medium CoDEx Small CoDEx Large WDsinger NELL23k FB15k237(10/20/50) AristoV4 DBpedia100k ConceptNet100k Hetionet
Benchmarks: FB15k-237 WN18RR CoDEx YAGO3-10 NELL-995 DBpedia100k ConceptNet100k Hetionet GraIL inductive splits ILPC 2022 InGram inductive datasets MTDEA datasets
Metrics: MRR Hits@10 Hits@10 (50 negs)
Baselines: NBFNet InGram ISDEA MTDEA RED-GNN A*Net NodePiece RotatE
Builds on: NBFNet DistMult InGram labeling trick AdamW
Closest papers by meaning
- Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs (NeurIPS 2025 · similarity 0.90)
- Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning (NeurIPS 2026 · similarity 0.89)
- Towards Global-Topology Relation Graph for Inductive Knowledge Graph Completion (AAAI 2025 · similarity 0.88)
- How Expressive are Knowledge Graph Foundation Models? (ICML 2025 · similarity 0.88)
- UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains (NeurIPS 2025 · similarity 0.87)
- A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.