Research map / Large language models / Retrieval and knowledge
Fast Think-on-Graph: Wider, Deeper and Faster Reasoning of Large Language Model on Knowledge Graph
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fast Think-on-Graph (FastToG), which has LLMs reason community by community over Wikidata using local community search with modularity-based coarse pruning and LLM-based fine pruning, plus Triple2Text and Graph2Text conversions.
Problem
Graph RAG over knowledge graphs struggles with complex multi-hop queries (narrow/shallow retrieval) and with high cost and recall loss when KGs are dense and reasoning chains are long.
Approach
Treat dense communities (found by local community detection on sampled subgraphs) as the steps of reasoning chains rather than single nodes or triples, prune candidates first by modularity and then by LLM selection, and convert community subgraphs to text via rule-based T2T or a fine-tuned T5 G2T model.
Key results
- With gpt-4o-mini, Ours(g2t) reaches 45.0% on CWQ and 65.8% on WebQSP, versus 42.9% and 63.6% for ToG (n-d 1-w).
- With Llama-3-70b, Ours(g2t) reaches 46.2% on CWQ and 94.5% on Creak, outperforming ToG on all six datasets except QALD.
- Using communities of max size 2 reduces average reasoning depth on CWQ by about 0.2 (t2t) and 0.4 (g2t) and on WebQSP by about 1.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FastToG with Graph2Text instead of the strongest listed baseline | gpt-4o-mini · CWQ | hit@1 accuracy (%): 42.9 -> 45.0 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | gpt-4o-mini · WebQSP | hit@1 accuracy (%): 64.4 -> 65.8 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | gpt-4o-mini · QALD | hit@1 accuracy (%): 54.9 -> 55.9 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | gpt-4o-mini · ZSRE | hit@1 accuracy (%): 54.0 -> 54.2 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | gpt-4o-mini · TREx | hit@1 accuracy (%): 64.2 -> 68.6 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | gpt-4o-mini · Creak | hit@1 accuracy (%): 95.4 -> 96.0 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | Llama-3-70b-instruct · CWQ | hit@1 accuracy (%): 40.3 -> 46.2 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | Llama-3-70b-instruct · WebQSP | hit@1 accuracy (%): 64.0 -> 66.4 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | Llama-3-70b-instruct · QALD | hit@1 accuracy (%): 51.6 -> 54.3 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | Llama-3-70b-instruct · ZSRE | hit@1 accuracy (%): 64.8 -> 67.9 | Helped |
| Use FastToG with Graph2Text instead of the strongest listed baseline | Llama-3-70b-instruct · TREx | hit@1 accuracy (%): 61.2 -> 64.7 | Helped |
| Use FastToG with Graph2Text instead of n-d 1-w | Llama-3-70b-instruct · Creak | hit@1 accuracy (%): 93.3 -> 94.5 | Helped |
Surprising findings
- Graph2Text is only marginally better than Triple2Text (mostly under 1%) and is slightly worse on QALD, attributed to hallucinations from the T5 G2T model that grow with community size.
- Community detection algorithm choice matters little (under 1% average difference), while random partitioning is over 3% worse.
Limitations
- Evaluated on a random 1k sample per dataset, which may limit statistical precision.
- G2T hallucination rises with community size, limiting gains from larger communities.
- Uses Wikidata with Neo4j and a fixed W=3 and D_max=5 setting, so generality to other KGs is untested.
Experimental setup
Datasets: CWQ WebQSP QALD ZSRE TREx Creak WebNLG Wikidata
Benchmarks: CWQ WebQSP QALD ZSRE TREx Creak
Metrics: exact match (hit@1) average reasoning depth number of LLM calls
Baselines: IO prompt CoT CoT-SC CoK KAPING KGP ToG
Builds on: Think-on-Graph Louvain Chain-of-Thought GraphRAG
Closest papers by meaning
- Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation (ICLR 2025 · similarity 0.92)
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.91)
- A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation (AAAI 2025 · similarity 0.90)
- Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.89)
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph (ICLR 2024 · similarity 0.89)
- GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.