Research map / Large language models / Retrieval and knowledge

Uncovering Overfitting in Large Language Model Editing

ICLR 2025 · Large language models › Retrieval and knowledge · approach: Knowledge editing in LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Identifies "Editing Overfit" in LLM knowledge editing, introduces the EVOKE benchmark with overfit metrics, and proposes Learn the Inference (LTI), a plug-and-play multi-stage inference constraint that reduces overfitting for ROME and MEMIT.

Problem

Knowledge editing methods that update LLM facts perform poorly on complex tasks such as multi-hop reasoning because edited models overfit, assigning disproportionately high probability to the edit target o* across unrelated prompts.

Approach

Attributes editing overfit to the paradigm of maximizing p(o*|p(s,r)) per edit sample; proposes LTI, which aligns the edited model's subject representations and output distributions with those of an unedited model that receives the new fact as in-context prefix, plus a new-knowledge loss.

Key results

• On GPT-J, ROME's direct probability on multi-hop reasoning is 41.03%, versus 0.27% for the unedited model; ROME-LTI reduces it to 11.12%.
• ROME-LTI raises multi-hop CAP for GPT-J from 17.96 (ROME) to 17.17 and EOS from 34.91 to 69.83, with lower DP on prefix distraction (26.13 to 8.93).
• Common overfitting mitigations (norm constraints, batch editing, paraphrase augmentation) are reported as ineffective or only marginally helpful, while specificity augmentation improves overfit tasks.

Measured effects reported in the paper

Surprising findings

• Paraphrase data augmentation made overfitting worse on most overfit tasks compared with plain MEMIT.
• Batch editing gave only marginal differences from single editing in overfitting behavior.

Limitations

• LTI relies on the unedited model's in-context learning ability to generate correct answers from context.
• LTI is designed for structured knowledge triples and may not extend to general knowledge editing.
• Results on Llama-2-7B show ROME-LTI and MEMIT-LTI slightly reduce paraphrase AMS.

Experimental setup

Datasets: CounterFact CounterFactPlus RippleEdits-Popular

Benchmarks: EVOKE

Metrics: Efficacy score (AMS) Paraphrase score (AMS) Direct Probability (DP) Correct Answer Probability (CAP) Editing Overfit Score (EOS) Answer Modify Score (AMS) Original Answer Probability (OAP)

Baselines: FT FT-L MEND ROME MEMIT

Builds on: ROME MEMIT MEND CounterFact CounterFactPlus

Closest papers by meaning

• Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing (ICML 2025 · similarity 0.92)
• UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.92)
• AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models (ICLR 2025 · similarity 0.91)
• Unveiling the Pitfalls of Knowledge Editing for Large Language Models (ICLR 2024 · similarity 0.91)
• Revealing and Mitigating Over-Attention in Knowledge Editing (ICLR 2025 · similarity 0.91)
• In-Context Editing: Learning Knowledge from Self-Induced Distributions (ICLR 2025 · similarity 0.91)