Research map / Large language models / Language, safety and interpretability
Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that approximate LLM unlearning methods can be reversed by fine-tuning on small, benign, loosely related public or partial-forget-set data ("benign relearning attacks"), across TOFU, WMDP, and Who's Harry Potter benchmarks.
Problem
Approximate LLM unlearning methods may only suppress outputs rather than truly remove knowledge, so unlearned content may be recovered by relearning on small benign data that does not directly contain the answers.
Approach
Relearning an unlearned model on auxiliary data that is correlated with, but does not contain direct answers to, the forgotten content can jog its memory of unlearned knowledge, unlike prior relearning studies that train on the unlearn data itself.
Key results
- On TOFU, relearning on one book per author recovers unseen keywords with over 70% attack success rate at shallow unlearning checkpoints.
- On WMDP with zephyr-7b-beta, gradient ascent unlearned forget score of 1.27 rises to 6.2 after relearning on GPT-4-generated public text.
- In the synthetic experiment, relearning on English text ending in 'Anthony' recovers 'Mark' with 100% ASR, versus 0% with no relearning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Relearn the GA-unlearned model on benign public text | zephyr-7b-beta · WMDP forget set | LLM-as-Judge score: 1.67 -> 5.2 | Helped |
| Relearn the GD-unlearned model on benign public text | zephyr-7b-beta · WMDP forget set | LLM-as-Judge score: 1.09 -> 5.2 | Helped |
| Relearn the NPO-unlearned model on benign public text | zephyr-7b-beta · WMDP forget set | LLM-as-Judge score: 1 -> 5.08 | Helped |
| Relearn the model with its limited unlearn-set subset, rather than training on that subset alone | Phi-1.5 · TOFU forget05 | ASR: 0% -> 10%, 40%, 30%, 20% | Helped |
| Add repetitions of the target token pair in training data | Llama-2-7b synthetic name-pair experiment · Synthetic name-pair experiment | relearn success rate: 1 repeated pair: 0%, 0%, 0%; 7 repeated pairs: 8%, 48%, 100% | Helped |
| Relearn with English text ending in Anthony rather than no relearning | Llama-2-7b synthetic experiment · Synthetic relevance experiment | ASR: 0% -> 100% | Helped |
| Relearn with English female names rather than no relearning | Llama-2-7b synthetic experiment · Synthetic relevance experiment | ASR: 0% -> 20% | Helped |
| Relearn with gibberish rather than no relearning | Llama-2-7b synthetic experiment · Synthetic relevance experiment | ASR: 0% -> 9% | Helped |
| Relearn with Japanese text ending in Anthony rather than no relearning | Llama-2-7b synthetic experiment · Synthetic relevance experiment | ASR: 0% -> 9% | Helped |
| Increase relearning beyond 40 steps in the TOFU experiment (ablation) | Unlearned TOFU models · TOFU | ASR | Hurt |
| Use public, related relearning data after WMDP gradient-ascent unlearning | WMDP gradient-ascent-unlearned model · WMDP forget set | LLM-as-Judge score: 1.27 -> 6.2 | Helped |
| Use the relearn set alone rather than an unlearned model with memorized knowledge | Phi-1.5 · TOFU forget05 | ASR: 10%, 40%, 30%, 20% -> 0%, 0%, 0%, 0% | Hurt |
Surprising findings
- Relearning on general public text, with no direct answers to the evaluation queries, restores unlearned hazardous and verbatim copyrighted content.
- Vanilla NLL fine-tuning is sufficient to recover unlearned content regardless of the unlearning objective used.
Limitations
- Attacks are evaluated only within the model parameter space with the same prompt; defenses such as prompt filtering, guardrails, or jailbreak combinations are not studied.
- Results depend on model choice, unlearning method, and evaluation metric (e.g., RMU was harder to relearn for sentence completion than for MCQ).
Experimental setup
Datasets: TOFU forget05 WMDP bio-attack corpus WMDP cyber-attack corpus Who's Harry Potter (WHP) text GPT-4-generated relearn text
Benchmarks: TOFU WMDP Who's Harry Potter (WHP)
Metrics: attack success rate (keyword match) LLM-as-judge score (1-10) Rouge-L F1 MT-Bench (retain) NLL on target tokens
Baselines: gradient ascent gradient difference KL minimization NPO SCRUB
Builds on: gradient ascent unlearning negative preference optimization RMU TOFU benchmark WMDP benchmark
Closest papers by meaning
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.94)
- Probing Hidden Knowledge Holes in Unlearned LLMs (NeurIPS 2025 · similarity 0.92)
- Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.92)
- Textual Unlearning Gives a False Sense of Unlearning (ICML 2025 · similarity 0.92)
- Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness (NeurIPS 2025 · similarity 0.92)
- Catastrophic Failure of LLM Unlearning via Quantization (ICLR 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.