Research map / Large language models / Language, safety and interpretability
Large Language Model Unlearning via Embedding-Corrupted Prompts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ECO Prompts, which gates prompts with a classifier and corrupts flagged prompt embeddings (first dimension, tuned by zeroth-order optimization offline) so that frozen LLMs approximate a retained model for unlearning.
Problem
LLM unlearning suffers from knowledge entanglement between forget and retain data and from the high compute cost of gradient-based unlearning on very large models.
Approach
Instead of updating weights, a prompt classifier flags forget-scope prompts and their input embeddings are corrupted with a scalar strength learned offline via zeroth-order finite-difference optimization, so the frozen LLM's outputs approximate a retained model's; cost is independent of model size.
Key results
- On WMDP, ECO reduces Zephyr-7B Bio/Chem/Cyber accuracy to 24.7/26.5/24.4 while keeping MMLU at 58.9 (unchanged from original).
- On MMLU subset unlearning with Zephyr-7B, ECO keeps econometrics, jurisprudence, and math at original accuracy (47.4, 74.1, 34.6) while dropping economics, law, and physics to near chance (20.6, 24.5, 23.1), whereas RMU hurts retained subjects.
- On copyrighted content (OLMo-7B), ECO achieves ASG 1.5 on BBC News and 2.1 on HP Book with utility unchanged (53.3 and 52.6).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ECO embedding-corrupted prompts instead of RMU | Zephyr-7B · WMDP Biology, Chemistry, and Cybersecurity | accuracy: 29.7 -> 24.7; 47.1 -> 26.5; 28.1 -> 24.4 | Helped |
| Use ECO rather than RMU for unlearning the MMLU target subjects | Zephyr-7B · MMLU Economics, Law, and Physics | accuracy: 27.3 -> 20.6; 27.8 -> 24.5; 27.0 -> 23.1 | Helped |
| Use ECO rather than RMU to retain related-subject performance | Zephyr-7B · MMLU Econometrics, Jurisprudence, and Math | accuracy: 41.2 -> 47.4; 37.0 -> 74.1; 29.2 -> 34.6 | Helped |
| Use ECO instead of the strongest reported ASG baseline | OLMo-7B fine-tuned on BBC News · BBC News | ASG: 3.9 -> 1.5 | Helped |
| Use ECO instead of the strongest reported ASG baseline | OLMo-7B fine-tuned on HP Book · HP Book | ASG: 1 -> 2.1 | Hurt |
| Use ECO instead of the MMLU original model | Zephyr-7B · MMLU full | accuracy: 58.9 -> 58.9 | No clear effect |
| Use random-noise corruption (ECO-RN) or zero-out corruption (ECO-ZO) (ablation) | ECO on TOFU · TOFU forget quality and model utility | forget quality | Helped |
| Use different tested corruption functions, including Gaussian noise or zeroing top entries (ablation) | ECO · Unlearning tasks | unlearning performance | No clear effect |
Surprising findings
- A simple random-noise or zero-out corruption of only the first embedding dimension, with one scalar tuned on Zephyr-7B, transfers to 100 models from 0.5B to 236B without per-model tuning.
- Prepending instruction-based guardrail prompts or in-context examples to prompts is noted to degrade regular task performance, in contrast to ECO's zero utility loss on general domains.
Limitations
- Requires API-only access; open-weight adversaries can bypass the classifier and corruption.
- Classifier context window is limited, so injected neutral text may evade detection unless a sliding-window approach is used.
- Does not defend against motivated adversaries who attack the classifier or the LLM itself.
Experimental setup
Datasets: TOFU WMDP MMLU Harry Potter and the Sorcerer's Stone BBC News
Benchmarks: TOFU WMDP MMLU BBC News Harry Potter book
Metrics: forget quality (KS p-value) model utility multiple-choice accuracy average similarity gap (ROUGE-L, SacreBLEU, BERTScore, METEO perplexity unique token ratio
Baselines: GA GD KL PO NPO LLMU SCRUB SSD RMU Prompting Mismatch
Builds on: negative preference optimization zeroth-order optimization RMU TOFU benchmark
Closest papers by meaning
- In-Context Unlearning: Language Models as Few-Shot Unlearners (ICML 2024 · similarity 0.92)
- Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting (NeurIPS 2025 · similarity 0.91)
- On Large Language Model Continual Unlearning (ICLR 2025 · similarity 0.91)
- RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models (NeurIPS 2024 · similarity 0.91)
- Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs (ICLR 2025 · similarity 0.91)
- A Closer Look at Machine Unlearning for Large Language Models (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.