Research map / Adversarial robustness and security / Attacks on text and graphs
Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DUA, an automated dynamic adversarial-suffix attack that recovers unlearned knowledge in LLMs, and LAU, a latent adversarial min-max unlearning framework yielding AdvGA and AdvNPO that are more robust to such attacks.
Problem
LLM unlearning methods are fragile: unlearned knowledge resurfaces under adversarial queries, and existing red-teaming relies on static, manually designed attack prompts.
Approach
Optimize universal GCG-style adversarial suffixes to reintroduce unlearned knowledge (DUA), then train unlearning under a latent-space min-max objective where perturbation vectors added to a middle residual layer simulate adversarial queries (LAU), applied to GA and NPO.
Key results
- DUA recovers unlearned knowledge on 55.2% of questions, even without access to the unlearned model's parameters.
- AdvNPO_KLR reduces the RWKU forget-set score by 57.0% relative to NPO_KLR on LLaMA-3-8B-Instruct, with a 10.6% drop on the neighbor set.
- On MUSE with LLaMA-2-7B-Chat, AdvGA lowers VerbMem on the forget set by 69.0% relative to GA (73.5 to 22.8 is reported as 69.0% reduction) while utility-preservation changes stay small.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add latent adversarial unlearning to NPO | NPO on RWKU · RWKU forget set, LLaMA-3-8B-Instruct | ROUGE-L All: 40.3 -> 15.5 | Helped |
| Add latent adversarial unlearning to GA | GA on RWKU · RWKU forget set, LLaMA-3-8B-Instruct | ROUGE-L All: 68.4 -> 57.2 | Helped |
| Add latent adversarial unlearning to NPO | NPO on RWKU · RWKU forget set, LLaMA-3.1-8B-Instruct | ROUGE-L All: 38.3 -> 18.7 | Helped |
| Add latent adversarial unlearning to GA | GA on RWKU · RWKU forget set, LLaMA-3.1-8B-Instruct | ROUGE-L All: 52.4 -> 30.2 | Helped |
| Add latent adversarial unlearning to NPO | NPO on MUSE news · MUSE news | KnowMem on forget set: 9.4 -> 6.1 | Helped |
| Add latent adversarial unlearning to GA | GA on MUSE news · MUSE news | KnowMem on forget set: 53.8 -> 41.7 | Hurt |
| Use gradient descent retain loss instead of the default retain loss (ablation) | AdvGA on RWKU · RWKU forget set, LLaMA-3-8B-Instruct | ROUGE-L All: 57.2 -> 62.6 | Hurt |
| Use KL-divergence retain loss instead of the default retain loss (ablation) | AdvGA on RWKU · RWKU forget set, LLaMA-3-8B-Instruct | ROUGE-L All: 57.2 -> 54.0 | Helped |
| Use gradient descent retain loss instead of the default retain loss (ablation) | AdvNPO on RWKU · RWKU forget set, LLaMA-3-8B-Instruct | ROUGE-L All: 15.5 -> 21.4 | Hurt |
| Use KL-divergence retain loss instead of the default retain loss (ablation) | AdvNPO on RWKU · RWKU forget set, LLaMA-3-8B-Instruct | ROUGE-L All: 15.5 -> 19.5 | Hurt |
| Add perturbations at the embedding layer (ablation) | Latent adversarial unlearning · RWKU forget and neighbor sets | ROUGE-L recall | No clear effect |
| Move the perturbation to deeper layers (ablation) | Latent adversarial unlearning · RWKU forget and neighbor sets | ROUGE-L recall | Hurt |
Surprising findings
- Attacks optimized on the original (pre-unlearning) model are nearly as effective as those optimized on the unlearned model, and even outperform them in the cross-query setting.
- Directly perturbing the embedding layer is ineffective; shallow (but not embedding) perturbation layers work best.
Limitations
- Robustness is only assessed against the proposed DUA attack and static prefix-injection prompts, not other jailbreak attacks.
- Some reported gains are inconsistent, e.g. AdvGA_GDR on MUSE/RWKU shows a neighbor-set increase in one case and a reported neighbor reduction of 11.6% vs 11.3% across the abstract and contribution list.
Experimental setup
Datasets: RWKU MUSE
Benchmarks: RWKU MUSE
Metrics: ROUGE-L forget score neighbor score MIA LOSS utility scores (reasoning, truthfulness, factuality, fluency VerbMem KnowMem PrivLeak
Baselines: GA NPO GA_GDR GA_KLR NPO_GDR NPO_KLR static prefix-injection attacks
Builds on: GCG negative preference optimization gradient ascent RWKU MUSE
Closest papers by meaning
- RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models (NeurIPS 2024 · similarity 0.92)
- Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond (ICML 2025 · similarity 0.91)
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.91)
- On Effects of Steering Latent Representation for Large Language Model Unlearning (AAAI 2025 · similarity 0.91)
- Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning (ICML 2025 · similarity 0.91)
- Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.