Research map / Large language models / AI and society
Position: Editing Large Language Models Poses Serious Safety Risks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing that knowledge editing methods for LLMs pose serious, overlooked safety risks because they are accessible, cheap, performant and stealthy, and that the AI ecosystem and institutions are unprepared.
Problem
Knowledge editing methods, designed to correct outdated facts in LLMs, can be repurposed by malicious actors to inject backdoors, bias, jailbreaks and misinformation, and current ecosystems do not verify model updates.
Approach
Argues that the properties making knowledge editing attractive (easy availability, low cost, high efficacy and specificity, stealth) make it a distinct safety threat beyond fine-tuning risks, and organizes the argument around attacker appeal, documented malicious use cases, ecosystem verification gaps, and awareness deficits, with a stakeholder vulnerability analysis and countermeasure agenda.
Key results
- ROME reports 99.8% Efficacy and 88.1% Generalization on GPT2-XL with zsRE, as cited to support KEs being performant.
- Editing 10 facts in LLaMA-7B takes under 7 seconds with MEND versus almost 170 seconds with MEMIT, as cited to support affordability.
- KE-based backdoor injection is reported as practical with as few as 15 poisoned samples and about 120s runtime, per cited prior work.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ROME to edit targeted facts | GPT2-XL · zsRE | Efficacy: 99.8% | Helped |
| Use ROME to edit targeted facts | GPT2-XL · zsRE | Generalization: 88.1% | Helped |
| Train the MEND hypernetwork, then edit 10 facts | LLAMA-7B · Editing 10 facts | editing time: less than 7 seconds | Helped |
| Edit 10 facts with MEMIT | LLAMA-7B · Editing 10 facts | editing time: almost 170 seconds | Mixed |
| Use a knowledge-editing framework to insert a backdoor | Decoder-only generative LLMs · Backdoor injection | poisoned samples required: as few as 15 poisoned samples | Helped |
| Use a knowledge-editing framework to insert a backdoor | Decoder-only generative LLMs · Backdoor injection | runtime: 120s | Helped |
Surprising findings
- Knowledge edits leave inference overhead unchanged and modify only a minimal fraction of parameters (e.g., ROME edits a single MLP matrix), making edited models hard to distinguish from unedited ones.
- Existing regulations such as the EU AI Act and California AI Transparency Act mostly focus on training data and process, leaving KE-based risks largely unaddressed.
Limitations
- Not an exhaustive survey of PEFT or knowledge editing methods, as the authors acknowledge.
- Countermeasures such as edit detection and reversal are limited in scope and rely on assumptions like available training data or model access.
- Analysis is limited to LLMs; multimodal editing attacks are only noted as future work.
Experimental setup
Datasets: zsRE
Benchmarks: zsRE
Metrics: efficacy generalization specificity perplexity
Baselines: LoRA DoRA MEND MALMEN IKE ROME MEMIT
Builds on: ROME MEMIT IKE MEND LoRA
Closest papers by meaning
- Unveiling the Pitfalls of Knowledge Editing for Large Language Models (ICLR 2024 · similarity 0.90)
- Should We Really Edit Language Models? On the Evaluation of Edited Language Models (NeurIPS 2024 · similarity 0.89)
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.89)
- Stealth edits to large language models (NeurIPS 2024 · similarity 0.88)
- BadEdit: Backdooring Large Language Models by Model Editing (ICLR 2024 · similarity 0.88)
- Concept-ROT: Poisoning Concepts in Large Language Models with Model Editing (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.