Research map / Large language models / Retrieval and knowledge
History Matters: Temporal Knowledge Editing in Large Language Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces temporal knowledge editing (TKE) and the AToKe benchmark, showing existing editors learn new facts but forget historical ones, and proposes METO (multi-editing with time objective) to retain history.
Problem
Existing knowledge editing methods overwrite outdated facts and cause catastrophic forgetting of historical knowledge, losing temporal ordering when facts change over time.
Approach
Treat outdated facts as historical knowledge to be preserved: edit both model-time historical facts and new current facts together, plus an extra objective that teaches the model the time span of each fact, applied on top of existing editors (MEND, ROME, MEMIT).
Key results
- Existing editors (ROME, MEMIT, MEND) reach ~98-100% current explicit accuracy after a single edit but historical explicit accuracy (HES) only ~2%.
- On AToKe-ME, HES* for edited models is below 0.3% for the baselines; with METO it rises to more than 15 (e.g., MEMIT+METO 21.93).
- METO keeps current-knowledge accuracy roughly stable for ROME and MEMIT, while greatly improving historical recall; overall AToKe remains difficult.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply METO to ROME | ROME on GPT-J (6B) · AToKe-SE | HES: 2.41 -> 20.25 | Helped |
| Apply METO to MEND | MEND on GPT-J (6B) · AToKe-SE | HRS: 0.68 -> 30.17 | Helped |
| Apply METO to MEMIT | MEMIT on GPT-J (6B) · AToKe-ME | HES*: 0.27 -> 21.93 | Helped |
| Apply METO to ROME | ROME on GPT-J (6B) · AToKe-ME | CES: 98.85 -> 99.93 | No clear effect |
| Edit the same fact multiple times rather than once | Knowledge editing methods · AToKe-ME versus AToKe-SE | HES: 0.44 versus 2.41 | Hurt |
| Use relative-time questions instead of explicit-time questions | Edited models · AToKe-SE | ROME historical score: 2.41 versus 1.56 | Hurt |
Surprising findings
- CFT is weak and hardly learns the new facts, unlike the locate-then-edit methods.
- Extending edits (EE) are easier than replacing objects, with current accuracy close to 100% for ROME and MEMIT.
Limitations
- Historical knowledge recall is still far from satisfactory even with METO.
- Only GPT-J (6B) is evaluated as the edited model.
- Benchmark relies on YAGO facts and synthetic counterfactual future facts generated by sampling.
Experimental setup
Datasets: YAGO3
Benchmarks: AToKe-SE AToKe-ME AToKe-EE
Metrics: CES CES-P CRS HES HRS HES*
Baselines: CFT MEND ROME MEMIT
Builds on: ROME MEMIT MEND
Closest papers by meaning
- WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (NeurIPS 2024 · similarity 0.91)
- Should We Really Edit Language Models? On the Evaluation of Edited Language Models (NeurIPS 2024 · similarity 0.90)
- Uncovering Overfitting in Large Language Model Editing (ICLR 2025 · similarity 0.90)
- Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models (AAAI 2025 · similarity 0.89)
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models (ICLR 2025 · similarity 0.89)
- Unveiling the Pitfalls of Knowledge Editing for Large Language Models (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.