Research map / Large language models / Instruction and fine-tuning
Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that a malicious user can disguise forgetting data by inserting benign trigger tokens (e.g., "please", "then") so that fine-tuning-based LLM unlearning degrades utility for normal users, and proposes Scope-aware Unlearning, which adds a scope term using mixed forget/retain prompts to restore robustness.
Problem
Fine-tuning-based LLM unlearning can be exploited: a malicious user submits a manipulated forgetting dataset so that benign tokens become unlearning signals, causing utility degradation for benign users of the unlearned model.
Approach
A Stealthy Attack inserts natural benign trigger templates into forget data to link common tokens with unlearning signals; the defense, Scope-aware Unlearning, adds a scope term that trains the model on concatenated forget/retain prompts to ignore unlearning signals in normal context.
Key results
- Stealthy Attack reduces LLaMA benign-trigger utility under GD at 10% forgetting from 0.683 to 0.160 (ROUGE-L) while clean utility stays about 0.70.
- Scope-aware Unlearning raises LLaMA GD benign-trigger utility under attack at 10% forgetting from 0.160 to 0.684 while keeping unlearning effectiveness near 0.2 ROUGE-L.
- On RWKU, SU recovers Mistral GD benign-trigger utility from 0.289 under attack to 0.797, close to the no-attack vanilla level of 0.589.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase benign-trigger token frequency in forgetting data with Stealthy Attack | Vanilla GD, NPO, and IDK unlearning · TOFU, 10% forgetting, LLaMA 3.1 8B | benign-trigger utility (ROUGE-L Recall): GD 0.683 -> 0.160; NPO 0.732 -> 0.217; IDK 0.496 -> 0.164 | Hurt |
| Increase benign-trigger token frequency in forgetting data with Stealthy Attack | Vanilla GD, NPO, and IDK unlearning · TOFU, 10% forgetting, Mistral v0.3 7B | benign-trigger utility (ROUGE-L Recall): GD 0.405 -> 0.056; NPO 0.556 -> 0.012; IDK 0.437 -> 0.192 | Hurt |
| Add the scope term to GD under SA | Vanilla GD unlearning · TOFU, 10% forgetting, LLaMA 3.1 8B | benign-trigger utility (ROUGE-L Recall): 0.160 -> 0.684 | Helped |
| Add the scope term to NPO under SA | Vanilla NPO unlearning · TOFU, 10% forgetting, LLaMA 3.1 8B | benign-trigger utility (ROUGE-L Recall): 0.217 -> 0.734 | Helped |
| Add the scope term to IDK under SA | Vanilla IDK unlearning · TOFU, 10% forgetting, LLaMA 3.1 8B | benign-trigger utility (ROUGE-L Recall): 0.164 -> 0.596 | Helped |
| Add the scope term to GD under SA | Vanilla GD unlearning · TOFU, 10% forgetting, Mistral v0.3 7B | benign-trigger utility (ROUGE-L Recall): 0.056 -> 0.216 | Helped |
| Add the scope term to NPO under SA | Vanilla NPO unlearning · TOFU, 10% forgetting, Mistral v0.3 7B | benign-trigger utility (ROUGE-L Recall): 0.012 -> 0.198 | Helped |
| Add the scope term to IDK under SA | Vanilla IDK unlearning · TOFU, 10% forgetting, Mistral v0.3 7B | benign-trigger utility (ROUGE-L Recall): 0.192 -> 0.373 | Helped |
| Add the scope term to GD with LoRA fine-tuning under SA (ablation) | Vanilla GD unlearning · TOFU, 10% forgetting, LoRA | benign-trigger utility (ROUGE-L Recall): 0.093 -> 0.458 | Helped |
| Add the scope term to NPO with LoRA fine-tuning under SA (ablation) | Vanilla NPO unlearning · TOFU, 10% forgetting, LoRA | benign-trigger utility (ROUGE-L Recall): 0.345 -> 0.763 | Helped |
| Add the scope term to IDK with LoRA fine-tuning under SA (ablation) | Vanilla IDK unlearning · TOFU, 10% forgetting, LoRA | benign-trigger utility (ROUGE-L Recall): 0.379 -> 0.685 | Helped |
Surprising findings
- LoRA fine-tuning is also highly susceptible: benign-trigger utility can fall below forget-data recall under GD and NPO.
- Continuous fine-tuning to recover benign-trigger utility substantially weakens unlearning effectiveness (forget ROUGE-L rises above 0.3).
Limitations
- Focuses only on fine-tuning-based unlearning methods; other unlearning approaches are not studied.
- Evaluated only on LLaMA 3.1 8B and Mistral v0.3 7B with TOFU and RWKU.
- Clean-utility gains and attack effects vary across settings; SU sometimes reduces unlearning effectiveness at 10% forgetting for GD on LLaMA (0.199 vs 0.008).
Experimental setup
Datasets: TOFU RWKU
Benchmarks: TOFU RWKU
Metrics: ROUGE-L Recall Probability Truth Ratio
Baselines: Gradient Difference NPO IDK Continuous fine-tuning Anomaly detection
Builds on: Gradient Difference NPO TOFU RWKU SimNPO
Closest papers by meaning
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning (ICLR 2025 · similarity 0.94)
- Textual Unlearning Gives a False Sense of Unlearning (ICML 2025 · similarity 0.93)
- Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.93)
- OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics (NeurIPS 2025 · similarity 0.93)
- LLM Unlearning via Loss Adjustment with Only Forget Data (ICLR 2025 · similarity 0.92)
- Large Language Model Unlearning (NeurIPS 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.