Research map / Adversarial robustness and security / Attacks on text and graphs
Erasing Conceptual Knowledge from Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Erasure of Language Memory (ELM), which uses the LM's own prefix-conditioned ("expert" vs "novice") next-token distributions to train low-rank adapters that erase biosecurity, cybersecurity, and Harry Potter knowledge.
Problem
Removing broad conceptual knowledge (e.g., dangerous biosecurity or cybersecurity knowledge) from LLMs without damaging general capabilities, while keeping generations coherent and resisting adversarial recovery of erased knowledge.
Approach
Use the language model itself as an implicit classifier: contrast next-token probabilities conditioned on an "expert" prefix versus a "novice" prefix, and fine-tune low-rank adapters on early layers so the model's distribution matches the classifier-modified distribution on concept text, with retain and fluency losses added.
Key results
- On Zephyr-7B, ELM reduces WMDP-Bio accuracy from 64.4 to 29.7 and WMDP-Cyber from 44.3 to 27.2 (near 25% random) while MMLU drops only from 58.5 to 56.6.
- ELM gets R-PPL 10.9 on Zephyr-7B versus 24.8 for RMU and 25.0 for RepNoise, indicating more fluent generations on erased concepts.
- On Harry Potter erasure with Llama-2-7B Chat, ELM lowers HP-MCQ from 66.4 to 38.3 while keeping MMLU at 45.3, versus WHP at 58.6 HP-MCQ.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ELM concept erasure | Zephyr-7B · WMDP-bio | accuracy: 64.4 -> 29.7 | Helped |
| Apply ELM concept erasure | Zephyr-7B · WMDP-cyber | accuracy: 44.3 -> 27.2 | Helped |
| Apply ELM concept erasure | Zephyr-7B · MMLU | accuracy: 58.5 -> 56.6 | Hurt |
| Apply ELM concept erasure | Zephyr-7B · R-PPL | reverse perplexity: 6.0 -> 10.9 | Hurt |
| Apply ELM instead of RMU | Zephyr-7B · WMDP-bio | accuracy: 30.5 -> 29.7 | No clear effect |
| Apply ELM instead of RMU | Zephyr-7B · R-PPL | reverse perplexity: 24.8 -> 10.9 | Helped |
| Apply ELM concept erasure | Llama-2-7B Chat · HP-MCQ | accuracy: 66.4 -> 38.3 | Helped |
| Apply ELM instead of WHP | Llama-2-7B Chat · HP-MCQ | accuracy: 58.6 -> 38.3 | Helped |
| Remove erase loss (ablation) | ELM on Zephyr-7B · WMDP-bio | accuracy: 29.7 -> 64.8 | Hurt |
| Remove retain loss (ablation) | ELM on Zephyr-7B · MMLU | accuracy: 56.6 -> 23.6 | Hurt |
| Remove fluency loss (ablation) | ELM on Zephyr-7B · R-PPL | reverse perplexity: 11.0 -> 29.8 | Hurt |
| Replace the erasure target with random logits (ablation) | ELM on Zephyr-7B · R-PPL | reverse perplexity: 11.0 -> 10.9 | Hurt |
Surprising findings
- For larger models (Qwen2.5-32B, Llama3-70B), the fluency loss is unnecessary and the erase loss alone maintains seamless generation.
- Full fine-tuning instead of low-rank adapters severely hurts MMLU (45.2 vs 56.6), and RMU failed to erase Harry Potter knowledge under the authors' sweep.
Limitations
- Degrades performance on semantically adjacent concepts (e.g., related MMLU classes).
- Generated text sometimes lacks semantic coherence, suggesting overly aggressive probability modification.
- Finetuning attack partially restores erased knowledge (Bio 29.7% to 42.2%).
Experimental setup
Datasets: WMDP-Bio forget corpus WMDP-Cyber forget corpus WMDP MCQ Harry Potter novel texts WikiText MMLU MT-Bench
Benchmarks: WMDP-Bio WMDP-Cyber MMLU MT-Bench HP-MCQ GCG attack BEAST attack
Metrics: MCQ accuracy MMLU accuracy MT-Bench score reverse perplexity (R-PPL) linear probe accuracy activation norms attack success
Baselines: RMU RepNoise WhoIsHarryPotter (WHP)
Builds on: Representation Misdirection for Unlearning (RMU) classifier-free guidance LoRA WhoIsHarryPotter
Closest papers by meaning
- Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.88)
- Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting (NeurIPS 2025 · similarity 0.88)
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning (ICLR 2025 · similarity 0.88)
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.88)
- Large Language Model Unlearning via Embedding-Corrupted Prompts (NeurIPS 2024 · similarity 0.88)
- Large Scale Knowledge Washing (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.