Research map / Graph learning / Knowledge and temporal graphs
GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GFM-RAG, an 8M-parameter query-dependent GNN graph foundation model that retrieves documents over an OpenIE-built knowledge graph index in a single step, pre-trained on 60 KGs and fine-tuned for retrieval, and evaluated on three multi-hop QA and seven domain RAG datasets.
Problem
Graph-enhanced RAG retrievers are hampered by noisy, incomplete KG structure, multi-step LLM-based retrieval is expensive, and existing GNN retrievers do not transfer to unseen corpora without retraining.
Approach
A query-dependent GNN (NBFNet-style DistMult messages, query/relation embeddings from a shared sentence encoder) is pre-trained with KG completion and then fine-tuned for document retrieval on 60 KG indexes, so it can perform multi-hop entity scoring in one pass on unseen KG indexes; entity scores are mapped to documents via an IDF-weighted inverted index.
Key results
- Retrieval R@2 on HotpotQA/MuSiQue/2Wiki reaches 78.3/49.1/90.8, outperforming IRCoT + HippoRAG by 16.8%, 8.3%, 19.8% in R@2 (as stated in text).
- Single-step GFM-RAG with GPT-4o-mini reader attains QA EM 51.6 on HotpotQA, 30.2 on MuSiQue, 69.8 on 2Wiki; with IRCoT reaches 56.0/36.6/72.5 EM.
- Retrieval time 0.107s on HotpotQA versus 3.162s for IRCoT + HippoRAG, while R@5 is highest on all three multi-hop sets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GFM-RAG instead of the strongest reported baseline | Multi-hop retrieval · HotpotQA, MuSiQue, and 2Wiki | R@2: 73.1 -> 78.3; 45.3 -> 49.1; 75.8 -> 90.8 | Helped |
| Use GFM-RAG instead of the strongest reported baseline | Multi-hop retrieval · HotpotQA, MuSiQue, and 2Wiki | R@5: 83.0 -> 87.1; 57.6 -> 58.2; 93.9 -> 95.6 | Helped |
| Add IRCoT multi-step retrieval to GFM-RAG | GFM-RAG · HotpotQA | EM: 51.6 -> 56.0 | Helped |
| Add IRCoT multi-step retrieval to GFM-RAG | GFM-RAG · MuSiQue | EM: 30.2 -> 36.6 | Helped |
| Add IRCoT multi-step retrieval to GFM-RAG | GFM-RAG · 2Wiki | EM: 69.8 -> 72.5 | Helped |
| Use GNN-based GFM retrieval instead of pre-trained or fine-tuned all-mpnet-v2 embeddings (ablation) | GFM-RAG · HotpotQA | R@2: 59.4 -> 78.3; 67.0 -> 78.3 | Helped |
| Remove supervised document retrieval fine-tuning (ablation) | GFM-RAG · HotpotQA | R@2: 78.3 -> 21.0 | Hurt |
| Remove supervised document retrieval fine-tuning (ablation) | GFM-RAG · MuSiQue | R@2: 49.1 -> 18.3 | Hurt |
| Remove supervised document retrieval fine-tuning (ablation) | GFM-RAG · 2Wiki | R@2: 89.1 -> 44.6 | Hurt |
| Remove KGC pre-training (ablation) | GFM-RAG · HotpotQA | R@2: 78.3 -> 77.8 | Hurt |
| Remove KGC pre-training (ablation) | GFM-RAG · MuSiQue | R@2: 49.1 -> 48.3 | Hurt |
| Remove KGC pre-training (ablation) | GFM-RAG · 2Wiki | R@2: 89.1 -> 88.3 | Hurt |
Surprising findings
- Removing KGC pre-training barely changes QA retrieval performance (e.g., HotpotQA R@2 77.8 vs 78.3), while KGC-only model performs poorly on the QA retrieval task.
- The model without retrieval fine-tuning scores much lower on retrieval (HotpotQA R@2 21.0) but best on KG completion MRR, showing trade-offs between tasks.
Limitations
- Relies on an LLM (GPT-4o-mini) for OpenIE during KG-index construction, adding indexing cost and dependence on a proprietary model.
- Document ranking relies on an explicit entity-to-document inverted index, which the authors note as a bridge that may limit end-to-end reasoning.
- Sentence embedding context limited to 512 tokens for the default all-mpnet-v2 encoder.
Experimental setup
Datasets: HotpotQA MuSiQue 2WikiMultiHopQA PubMedQA DelucionQA TechQA ExpertQA EManual MS Marco HAGRID
Benchmarks: HotpotQA MuSiQue 2WikiMultiHopQA PubMedQA DelucionQA TechQA ExpertQA EManual MS Marco HAGRID
Metrics: recall@2 recall@5 EM F1 retrieval time (s) MRR
Baselines: BM25 Contriever GTR ColBERTv2 RAPTOR Proposition GraphRAG (MS) LightRAG HippoRAG SubgraphRAG G-retriever Adaptive-RAG
Builds on: NBFNet ULTRA HippoRAG IRCoT DistMult
Closest papers by meaning
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.92)
- A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation (AAAI 2025 · similarity 0.91)
- Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need? (NeurIPS 2025 · similarity 0.91)
- RAGraph: A General Retrieval-Augmented Graph Learning Framework (NeurIPS 2024 · similarity 0.91)
- Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation (ICLR 2025 · similarity 0.90)
- RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.