Research map / Large language models / Retrieval and knowledge
Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IFMET, a two-stage locate-then-edit method that edits shallow MLPs with single-hop prompts and deeper MLPs with generated multi-hop edit prompts, improving multi-hop factual recall after knowledge editing on MQuAKE-3K.
Problem
Locate-then-edit knowledge editing methods succeed on single-hop recall but fail on multi-hop factual recall that uses newly edited facts, because they only edit shallow MLP layers with single-hop prompts.
Approach
Mechanistic analysis (LogitLens and causal intervention) shows implicit subjects in multi-hop queries are retrieved from deeper MLPs at the last token. IFMET adds a furtherance edit stage that uses constructed multi-hop edit prompts to modify deeper MLP layers, on top of the standard single-hop shallow edit.
Key results
- On MQuAKE-3K with CoT, IFMET reaches 31.01 average multi-hop accuracy versus 17.04 for PMET (GPT-J-6B).
- In the ablation on MQuAKE-CF subset, IFMET multi-hop accuracy is 28.38 versus 15.94 for PMET, a 78% relative gain.
- Removing the deeper-layer furtherance edit drops multi-hop accuracy to 15.07, below PMET's 15.94.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply IFMET+CoT instead of PMET+CoT | GPT-J-6B · MQuAKE-3K, few-shot multi-hop factual recall | average accuracy: 17.04 -> 31.01 | Helped |
| apply IFMET+CoT instead of PMET+CoT | GPT-J-6B · MQuAKE-3K, edited answer, Post | accuracy: 11.91 -> 35.15 | Helped |
| add Chain-of-Thought prompting to IFMET | IFMET on MQuAKE-3K · MQuAKE-3K, edited answer, average | accuracy: 23.04 -> 31.01 | Helped |
| add the first-stage single-hop edit (ablation) | IFMET on MQuAKE-CF subset · MQuAKE-CF subset, 1 edit | efficacy: 66.59 -> 99.56 | Helped |
| add the first-stage single-hop edit (ablation) | IFMET on MQuAKE-CF subset · MQuAKE-CF subset, 1 edit | multi-hop accuracy: 23.14 -> 28.38 | Helped |
| use a multi-hop prompt rather than a single-hop prompt in the furtherance stage (ablation) | IFMET on MQuAKE-CF subset · MQuAKE-CF subset, 1 edit | multi-hop accuracy: 17.69 -> 28.38 | Helped |
| edit deeper rather than shallow layers in the furtherance stage (ablation) | IFMET on MQuAKE-CF subset · MQuAKE-CF subset, 1 edit | multi-hop accuracy: 15.07 -> 28.38 | Helped |
Surprising findings
- Using single-hop edit prompts for the deeper-layer edit (w/o Multi) yields almost no multi-hop gain (17.69 vs 15.94 for PMET), indicating single-hop prompts cannot modify deep-layer knowledge.
- Editing deeper layers with multi-hop prompts but in shallow layers (w/o Deeper) gives no improvement over PMET, so layer location matters with the prompt type.
Limitations
- Only evaluated on GPT-J-6B in the main experiments; generalization to newer models is only briefly addressed in appendix.
- Multi-hop edit prompts depend on constructing supporting triples from WikiData or the model itself, adding preprocessing cost.
- Relies on PMET as the underlying editor for both stages.
Experimental setup
Datasets: MQuAKE-3K MQuAKE-CF WikiData Wikitext COUNTERFACT
Benchmarks: MQuAKE-3K MQuAKE-CF
Metrics: multi-hop accuracy efficacy paraphrase specificity
Baselines: FT MEND ROME MEMIT PMET
Builds on: ROME MEMIT PMET LogitLens
Closest papers by meaning
- Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs (ICLR 2025 · similarity 0.88)
- Uncovering Overfitting in Large Language Model Editing (ICLR 2025 · similarity 0.88)
- Joint Knowledge Editing for Information Enrichment and Probability Promotion (AAAI 2025 · similarity 0.88)
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models (ICLR 2025 · similarity 0.87)
- WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (NeurIPS 2024 · similarity 0.87)
- Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.