Research map / Large language models / Instruction and fine-tuning
Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ILU, an IRM-inspired invariance regularizer added to NPO/RMU LLM unlearning so that forgetting survives unrelated downstream fine-tuning, evaluated on WMDP and MUSE with Zephyr-7B-beta and LLaMA3-8B-Instruct.
Problem
LLM unlearning is fragile: knowledge erased by existing methods (NPO, RMU) can be rapidly recovered by fine-tuning on unrelated downstream tasks or by relearning attacks.
Approach
Introduce IRMv1-style invariance regularization (gradient norm of the loss with respect to a dummy scalar predictor) on a single unrelated fine-tuning dataset (e.g., GSM8K) added to the NPO or RMU unlearning objective, so the unlearned model stays stationary under fine-tuning perturbations; analyzed with task vectors.
Key results
- RMU+ILU(GSM8K) raises average robust accuracy after six downstream fine-tunings to 0.65 vs 0.42 for RMU; NPO+ILU reaches 0.56 vs 0.47 for NPO.
- On MUSE-News, NPO VerbMem rises from 2.53 to 57.27 after WinoGrande fine-tuning, while NPO+ILU(GSM8K) stays at 0.00 across fine-tuning settings.
- With LLaMA3-8B-Instruct, NPO+ILU matches TAR's average robust accuracy (0.70) at about 118 minutes vs 7441.9 minutes for TAR (roughly 63x cheaper).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add invariance regularization to RMU | RMU on WMDP · WMDP, post-finetune average | robust accuracy: 0.42 -> 0.65 | Helped |
| Add invariance regularization to NPO | NPO on WMDP · WMDP, post-finetune average | robust accuracy: 0.47 -> 0.56 | Helped |
| Add invariance regularization to NPO | NPO on WMDP · WMDP, pre-finetune | forget quality: 0.52 -> 0.56 | Helped |
| Add GSM8K-based invariance regularization to NPO | NPO on MUSE-News · MUSE-News, post-finetune on WinoGrande | VerbMem: 57.27 -> 0.00 | Helped |
| Use one unrelated invariance dataset (GSM8K) rather than multiple datasets (ablation) | ILU with RMU objective · WMDP, downstream fine-tuning | robustness | Helped |
| Use the forget set (WMDP) instead of unrelated GSM8K for invariance regularization (ablation) | ILU with RMU objective · WMDP, downstream fine-tuning | forget quality | Hurt |
| Use multiple invariance datasets (GSM8K, AGNews, and WinoGrande) instead of one unrelated dataset (ablation) | ILU with RMU objective · WMDP, post-finetune average | robust accuracy: 0.65 -> 0.64 | Hurt |
| Use multiple invariance datasets (GSM8K, AGNews, and WinoGrande) instead of one unrelated dataset (ablation) | ILU with NPO objective · WMDP, post-finetune average | robust accuracy: 0.56 -> 0.52 | Hurt |
| Increase the invariance regularization parameter to an overly large value (e.g., λ > 0.1) (ablation) | ILU(GSM8K) · WMDP | forget quality | Hurt |
| Set the invariance regularization parameter too low (e.g., λ = 0.05) (ablation) | ILU(GSM8K) · WMDP, downstream fine-tuning | robustness | Hurt |
| Use GSM8K invariance regularization rather than SAM-based sharpness optimization | NPO+ILU vs NPO+SAM · WMDP, relearning attack | forget quality drop: 0.02 vs 0.06 | Mixed |
| Add GSM8K-based invariance regularization to NPO | NPO on WMDP · WMDP, LLaMA3-8B-Instruct post-finetune average | robust accuracy: 0.61 -> 0.70 | Helped |
Surprising findings
- Using the forget set itself for invariance regularization (ILU(WMDP)) underperforms using an unrelated dataset, because the forget-set regularizer conflicts with the unlearning objective.
- Adding multiple fine-tuning sets (ILU(Multi)) gives no robustness advantage over a single unrelated set and can destabilize MMLU for NPO.
Limitations
- Under relearning attacks, ILU's gains are smaller than SAM-based defenses (e.g., NPO+ILU drop 0.06 vs NPO+SAM 0.02).
- Robust accuracy is based on a selected set of fine-tuning tasks and epochs; theoretical understanding of invariance-based unlearning is left for future work.
- Hyperparameter lambda is sensitive: too large harms forget quality, too small loses robustness.
Experimental setup
Datasets: WMDP WikiText GSM8K AGNews WinoGrande SST-2 MNLI QQP MUSE
Benchmarks: WMDP MUSE MMLU
Metrics: forget quality robust accuracy fine-tuning accuracy MMLU accuracy VerbMem KnowMem running time
Baselines: NPO RMU ILU(Multi) LAT TAR SAM-based unlearning
Builds on: NPO RMU IRMv1 task vectors
Closest papers by meaning
- Unified Parameter-Efficient Unlearning for LLMs (ICLR 2025 · similarity 0.92)
- OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics (NeurIPS 2025 · similarity 0.92)
- Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.92)
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.92)
- Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs (ICLR 2025 · similarity 0.91)
- Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.