Research map / Large language models / Retrieval and knowledge
Should We Really Edit Language Models? On the Evaluation of Edited Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Evaluates how sequential knowledge editing (ROME, MEMIT, PMET, MEND, KN, GRACE, SERAC-style methods) degrades general abilities and safety of LLMs across edit counts, model type, and scale.
Problem
Model editing is proposed as an efficient way to update knowledge in LLMs, but its effect on general abilities (world knowledge, math, reasoning, reading comprehension, safety) of sequentially edited models is unexplored.
Approach
A systematic empirical study that sequentially edits Llama2, Mistral, GPT2-XL and Pythia models with several editing methods on CounterFact and measures general-ability and safety benchmarks as a function of edit count, instruction tuning, model scale, and benchmark type, finding that most methods degrade after only dozens of edits and that a 'muting effect' appears at very large edit counts.
Key results
- Most editing methods keep general abilities within dozens of edits; ROME and MEMIT on Llama2-7B collapse to near zero by 100-500 edits, while PMET and MEND stay stable over hundreds to 1000 edits.
- At 10k sequential edits, ROME, MEMIT and PMET on Llama, Mistral and GPT2-XL produce empty strings or random characters (the 'muting effect').
- Even a few dozen edits reduce TruthfulQA and ToxiGen scores for edited Llama2-7B and Mistral-7B models, including safety-aligned chat models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase ROME edits from 0 to 100 | Llama2-7B · MMLU | accuracy: 0.4587 -> 0.0007 | Hurt |
| Increase MEMIT edits from 0 to 100 | Llama2-7B · MMLU | accuracy: 0.4587 -> 0.4428 | Hurt |
| Use PMET with 100 edits rather than no editing | Llama2-7B · MMLU | accuracy: 0.4587 -> 0.4593 | No clear effect |
| Use MEND with 1000 edits rather than no editing | Llama2-7B · MMLU | accuracy: 0.4587 -> 0.4571 | No clear effect |
| Apply 10 KN edits rather than no editing | Llama2-7B · MMLU | accuracy: 0.4587 -> 0 | Hurt |
| Increase ROME edits from 0 to 50 | Llama2-7B-Chat · MMLU | accuracy: 0.4516 -> 0.2700 | Hurt |
| Use MEMIT with 100 edits rather than no editing | Llama2-7B-Chat · MMLU | accuracy: 0.4516 -> 0.4485 | No clear effect |
| Increase ROME edits from 0 to 100 | Mistral-7B · MMLU | accuracy: 0.6233 -> 0 | Hurt |
| Use PMET with 10 edits rather than no editing | Mistral-7B · MMLU | accuracy: 0.6233 -> 0.6204 | No clear effect |
| Increase ROME edits from 0 to 10 | Pythia-160M · MMLU | accuracy: 0.2435 -> 0 | Hurt |
| Use MEMIT with 100 edits rather than no editing | Pythia-160M · MMLU | accuracy: 0.2435 -> 0.2468 | No clear effect |
| Increase MEMIT edits from 0 to 100 | Llama2-7B · TruthfulQA | accuracy: 0.2521 -> 0.1922 | Hurt |
Surprising findings
- Some methods show improved safety scores at 100 edits, which the authors attribute to disruption of the model's knowledge structure rather than genuine improvement.
- Instruction-tuned models declined more slowly under MEMIT editing, but many other results showed comparable degradation between base and instruction-tuned models.
Limitations
- Only six editing methods and four to seven benchmarks evaluated.
- Limited deeper analysis of why degradation occurs.
- Hyperparameters per method were not tuned extensively; only models supported by each method were evaluated, and some results are missing for Mistral.
Experimental setup
Datasets: CounterFact ZsRE
Benchmarks: MMLU BBH GSM8K CommonsenseQA TriviaQA TruthfulQA ToxiGen
Metrics: accuracy exact match solve rate
Baselines: ROME MEMIT PMET MEND KN GRACE
Builds on: ROME MEMIT MEND GRACE PMET
Closest papers by meaning
- Unveiling the Pitfalls of Knowledge Editing for Large Language Models (ICLR 2024 · similarity 0.92)
- Reasons and Solutions for the Decline in Model Performance after Editing (NeurIPS 2024 · similarity 0.92)
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models (ICLR 2025 · similarity 0.91)
- Perturbation-Restrained Sequential Model Editing (ICLR 2025 · similarity 0.91)
- UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models (NeurIPS 2025 · similarity 0.90)
- Uncovering Overfitting in Large Language Model Editing (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.