Research map / Large language models / Retrieval and knowledge
Knowledge graph reasoning with LLMs
Ground LLM reasoning in knowledge graphs through logic-aware and graph-constrained methods.
174 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Retrieval and knowledge, keyword cluster “question · answering · editing”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 4 of 4 tracked benchmarks.
31 papers in 2023–24 and 79 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 13.6% to 12.0%.
Open questions the data points to
- WebQSP (Hits@1): best went from 87.3 (up to 2024) to 88.8 in 2025 — EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering (AAAI 2025 · 5 citations)
- CWQ (Hits@1): best went from 75 (up to 2024) to 79.5 in 2025 — Promoting Knowledge Base Question Answering by Directing LLMs to Generate Task-relevant Logical Forms (AAAI 2025 · 4 citations)
- CWQ (F1): best went from 56.2 (up to 2024) to 70.5 in 2025 — Deliberation on Priors: Trustworthy Reasoning of Large Language Models on Knowledge Graphs (NeurIPS 2025 · 1 citation)
- WebQSP (F1): best went from 70.8 (up to 2024) to 84.24 in 2025 — Promoting Knowledge Base Question Answering by Directing LLMs to Generate Task-relevant Logical Forms (AAAI 2025 · 4 citations)
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · A Joint Model for Question Answering over Multiple Knowledge Bases (AAAI 2016 · 38 citations)
- 2017 · Unit Dependency Graph and Its Application to Arithmetic Word Problem Solving (AAAI 2017 · 90 citations)
- 2018 · Variational Reasoning for Question Answering With Knowledge Graph (AAAI 2018 · 394 citations)
- 2019 · ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning (AAAI 2019 · 768 citations)
- 2020 · K-BERT: Enabling Language Representation with Knowledge Graph (AAAI 2020 · 794 citations)
- 2021 · (Comet-) Atomic 2020: On Symbolic and Neural Commonsense Knowledge Graphs (AAAI 2021 · 293 citations)
- 2022 · StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts (AAAI 2022 · 34 citations)
- 2023 · Enabling Knowledge Refinement upon New Concepts in Abductive Learning (AAAI 2023 · 12 citations)
- 2024 · Knowledge Graph Prompting for Multi-Document Question Answering (AAAI 2024 · 155 citations)
- 2025 · Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · 24 citations)
- 2026 · ProgRAG: Hallucination-Resistant Progressive Retrieval and Reasoning over Knowledge Graphs (AAAI 2026 · 5 citations)
Earliest papers here
- A Joint Model for Question Answering over Multiple Knowledge Bases (AAAI 2016 · 38 citations)
- Explaining Inconsistency-Tolerant Query Answering over Description Logic Knowledge Bases (AAAI 2016 · 32 citations)
- Ontology-Mediated Queries for NOSQL Databases (AAAI 2016 · 14 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
WebQSP · Hits@1 (higher is better)
7 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 51.9 | Reasoning Over Virtual Knowledge Bases With Open Predicate Relations (ICML 2021 · 14 citations) |
| 2024 | 87.3 | Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs (NeurIPS 2024 · 38 citations) |
| 2025 | 88.8 | EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering (AAAI 2025 · 5 citations) |
CWQ · Hits@1 (higher is better)
6 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 29.4 | LEGO: Latent Execution-Guided Reasoning for Multi-Hop Question Answering on Knowledge Graphs (ICML 2021 · 31 citations) |
| 2024 | 75 | Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs (NeurIPS 2024 · 38 citations) |
| 2025 | 79.5 | Promoting Knowledge Base Question Answering by Directing LLMs to Generate Task-relevant Logical Forms (AAAI 2025 · 4 citations) |
CWQ · F1 (higher is better)
5 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 56.2 | Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · 40 citations) |
| 2025 | 70.5 | Deliberation on Priors: Trustworthy Reasoning of Large Language Models on Knowledge Graphs (NeurIPS 2025 · 1 citation) |
WebQSP · F1 (higher is better)
5 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 70.8 | Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · 40 citations) |
| 2025 | 84.24 | Promoting Knowledge Base Question Answering by Directing LLMs to Generate Task-relevant Logical Forms (AAAI 2025 · 4 citations) |
What has been tried
109 papers here report 1,064 measured changes (a component added, removed or swapped, compared with a baseline): helped 76%, no clear effect 6%, hurt 15%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use T5 instead of MDR as the KG traversal agent | HotpotQA · F1 | 65.77 -> 66.77 | Helped | Knowledge Graph Prompting for Multi-Document Question Answering (AAAI 2024 · 155 citations) |
| add KG-grounded relation-path planning | WebQSP · F1 | 49.69 -> 70.81 | Helped | Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · 40 citations) |
| add Guidance (question decomposition into sub-objectives) | CWQ · Hits@1 | 60.1 -> 63.2 | Helped | Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs (NeurIPS 2024 · 38 citations) |
| Remove subgraph focusing and question simplifying | MetaQA 2-hop · accuracy | 96.6 -> 76.6 | Hurt | Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · 24 citations) |
| add structural information to knowledge embedding | WebQSP · Hits@1 | 82.36 -> 83.77 | Helped | LightPROF: A Lightweight Reasoning Framework for Large Language Model on Knowledge Graph (AAAI 2025 · 22 citations) |
| replace parallel editing with progressive dynamic knowledge adapting | HotpotQA (3-shot) · exact match | 31.2% -> 34.1% | Helped | Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources (ICLR 2024 · 18 citations) |
| Provide one question-related relation as a reference | GrailQA I.I.D and compositional · F1 | No clear effect | Code-Style In-Context Learning for Knowledge-Based Question Answering (AAAI 2024 · 17 citations) | |
| add current-data rules to historical-data rules | ICEWS14 · MRR | 0.450 -> 0.457 | Helped | Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning (NeurIPS 2024 · 17 citations) |
| Add binary tree decomposition to instruction tuning | FB15K hard queries, including 3p and 3i · accuracy | 60.2 -> 65.6; 44.8 -> 75.2 | Helped | Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning (AAAI 2025 · 12 citations) |
| remove add-column operation (adding it back) | TabFact · accuracy | 85.23 -> 86.61 | Helped | Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding (ICLR 2024 · 12 citations) |
| Add a global edge to connect the global node with fact-unit nodes | ReClor Dev · accuracy | 64.6 -> 66.8 | Helped | Fact-Driven Logical Reasoning for Machine Reading Comprehension (AAAI 2024 · 11 citations) |
| Add Topic Prune | WebQSP · EM | 77.6% -> 81.1% | Helped | Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation (ICLR 2025 · 9 citations) |
What reviewers will expect
Datasets and baselines named in the 36 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- WebQSP · 12 of 36 papers
- CWQ · 8 of 36 papers
- Freebase · 5 of 36 papers
- FB15k-237 · 3 of 36 papers
- NELL995 · 3 of 36 papers
- MetaQA · 3 of 36 papers
- GrailQA · 3 of 36 papers
- Wikidata · 3 of 36 papers
Baselines
- ToG · 9 of 36 papers
- UniKGQA · 8 of 36 papers
- RoG · 7 of 36 papers
- StructGPT · 5 of 36 papers
- EmbedKGQA · 4 of 36 papers
- NSM · 4 of 36 papers
- TransferNet · 4 of 36 papers
- KD-CoT · 4 of 36 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 110 papers; from OpenAlex and the papers' affiliations).
- Hong Kong University of Science and Technology · 7 papers
- Nanjing University · 5 papers
- University of Science and Technology of China · 4 papers
- Nanyang Technological University · 4 papers
- Shanghai Jiao Tong University · 4 papers
- Xi’an Jiaotong University · 4 papers
- Monash University · 4 papers
- Griffith University · 4 papers
- Chinese Academy of Sciences · 3 papers
- Alibaba Group · 3 papers
Use it
Download BibTeX of 37 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Knowledge editing in LLMs (170 papers)
- Knowledge-augmented question answering (134 papers)
- Explanatory and multiple-choice QA benchmarks (152 papers)
- Domain-specific QA datasets (79 papers)
Elsewhere in the topic
- RAG robustness and evaluation (164 papers)
- Retrieval-augmented reasoning (130 papers)
- Long-context and personalized retrieval (61 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).