Research map / Large language models / Retrieval and knowledge
EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EPERM, a three-stage LLM-based KGQA framework (subgraph retrieval, weighted evidence path finding, answer prediction) jointly fine-tuned from LLaMA2-Chat-7B under a graphical-model/ELBO formulation, reporting state-of-the-art Hits@1 on WebQSP and CWQ.
Problem
LLM-based knowledge graph question answering treats retrieved KG information equally and decouples retrieval from reasoning, ignoring differing importance of evidence paths.
Approach
Reformulates KGQA as a directed probabilistic graphical model with latent subgraph and evidence paths; an LLM generates weighted relation plans, scores and filters evidence paths in the retrieved subgraph, and a jointly fine-tuned answer predictor reasons over the weighted paths.
Key results
- EPERM achieves 88.8 Hits@1 and 72.4 F1 on WebQSP, versus 85.7 Hits@1 for the prior best RoG.
- EPERM achieves 66.2 Hits@1 and 58.9 F1 on CWQ, versus 62.6 Hits@1 for RoG.
- Removing the evidence path finder drops WebQSP Hits@1 from 88.8 to 66.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add evidence-path scoring and filtering (ablation) | EPERM · WebQSP | Hits@1: 84.2 -> 88.8 | Helped |
| add evidence-path scoring and filtering (ablation) | EPERM · WebQSP | F1: 69.1 -> 72.4 | Helped |
| add evidence-path scoring and filtering (ablation) | EPERM · CWQ | Hits@1: 61.3 -> 66.2 | Helped |
| add evidence-path scoring and filtering (ablation) | EPERM · CWQ | F1: 55.8 -> 58.9 | Helped |
| add the evidence path finder (ablation) | EPERM · WebQSP | Hits@1: 66.2 -> 88.8 | Helped |
| add the evidence path finder (ablation) | EPERM · CWQ | Hits@1: 36.8 -> 66.2 | Helped |
| add the answer predictor (ablation) | EPERM · WebQSP | Hits@1: 62.3 -> 88.8 | Helped |
| add the answer predictor (ablation) | EPERM · CWQ | Hits@1: 33.1 -> 66.2 | Helped |
| remove the answer predictor and use all weighted-path answers (ablation) | EPERM · WebQSP | Recall: 76.4 -> 79.8 | Helped |
| remove the answer predictor and use all weighted-path answers (ablation) | EPERM · CWQ | Recall: 60.9 -> 66.2 | Helped |
Surprising findings
- Removing the answer predictor and returning all answers from weighted evidence paths raises recall (76.4 to 79.8 on WebQSP) but sharply lowers Hits@1 (88.8 to 62.3).
Limitations
- Relies on the Freebase knowledge graph and topic entity linking assumed given, limiting generality.
- Evaluated only on two benchmarks with a single 7B backbone.
Experimental setup
Datasets: WebQSP CWQ Freebase
Benchmarks: WebQSP CWQ
Metrics: Hits@1 F1 Recall
Baselines: KV-Mem EmbedKGQA NSM TransferNet KGT5 BAMnet GraftNet PullNet SR+NSM SR+NSM+E2E EPR+NSM LLaMA2-Chat-7B
Builds on: RoG UniKGQA ToG
Closest papers by meaning
- Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · similarity 0.92)
- Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models (ICML 2025 · similarity 0.90)
- Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · similarity 0.89)
- Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking (NeurIPS 2025 · similarity 0.88)
- Graph Reasoning Transformers for Knowledge-Aware Question Answering (AAAI 2024 · similarity 0.88)
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.