Research map / Large language models / Retrieval and knowledge

Knowledge Editing with Dynamic Knowledge Graphs for Multi-Hop Question Answering

AAAI 2025 · Large language models › Retrieval and knowledge · approach: Knowledge graph reasoning with LLMs · 8 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes KeDkg, a memory-based knowledge editing method that stores edited facts as a dynamic knowledge graph, resolves secondary-edit conflicts, and uses a fine-tuned LLM for question decomposition plus entity/relation detectors for graph-augmented answering on MQuAKE.

Problem

Knowledge editing for multi-hop question answering in LLMs suffers from inaccurate retrieval of edited facts, cascading effects across hops, and conflicts arising from secondary (repeated) edits.

Approach

Convert edited texts into triples stored in a dynamic knowledge graph (with Wikidata entity linking and conflict-based replacement of old edits), decompose multi-hop questions with a fine-tuned LLM, and retrieve facts via a trained entity detector and relation detector while keeping the base LLM frozen.

Key results

• On MQuAKE-CF-3K with Llama 2-7B at 1 edit, KeDkg reaches 66.80 M-Acc and 63.67 H-Acc, versus 44.13 / 30.60 for PokeMQA (the best prior baseline).
• Under the all-edited setting on MQuAKE-CF-3K with Llama 2-7B, KeDkg achieves 48.30 M-Acc and 43.90 H-Acc, compared with 32.83 / 23.87 for PokeMQA.
• Removing the conflict detection and modification module lowers all-edited M-Acc on MQuAKE-CF-3K from 48.30 to 41.10 (reported as a 7.2% drop).

Measured effects reported in the paper

Surprising findings

• Parameter-based editing methods (ROME, MEMIT, FT) underperform parameter-preserving methods on multi-hop QA, with notably low hop-wise accuracy.
• Removing the entity detector or relation detector in favor of all-MiniLM-L6-v2 similarity hurts performance, with the relation detector causing the largest drop.

Limitations

• Relies on a fine-tuned question decomposition LLM and trained detectors, so performance depends on the training data distribution (MQuAKE-CF).
• Evaluated only on MQuAKE benchmarks (CF-3K and T); generalization to other MHQA or editing datasets is not shown.
• Accuracy still drops to about 48% under all-edited setting on MQuAKE-CF-3K.

Experimental setup

Datasets: MQuAKE-CF MQuAKE-T Wikidata

Benchmarks: MQuAKE-CF-3K MQuAKE-T

Metrics: Multi-hop accuracy (M-Acc) Hop-wise answering accuracy (H-Acc)

Baselines: MeLLo PokeMQA FT ROME MEMIT

Builds on: MeLLo PokeMQA ROME MEMIT chain-of-thought

Closest papers by meaning

• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation (AAAI 2025 · similarity 0.90)
• Question Calibration and Multi-Hop Modeling for Temporal Question Answering (AAAI 2024 · similarity 0.89)
• Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · similarity 0.89)
• Knowledge Graph Prompting for Multi-Document Question Answering (AAAI 2024 · similarity 0.89)
• Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.88)
• Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking (NeurIPS 2025 · similarity 0.88)