Research map / Large language models / Language, safety and interpretability
Distillation Robustifies Unlearning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that distilling an unlearned model into a randomly initialized student (and a noised-copy variant, UNDO) yields unlearning that resists relearning attacks, approaching data-filtering oracle robustness at lower cost.
Problem
Current LLM unlearning methods only suppress capabilities; a few finetuning steps can restore them, so robust capability removal is unsolved.
Approach
Distilling a suppressed (unlearned) model via forward KL into a randomly initialized or partially noised copy transfers desired behavior while leaving latent undesired capabilities behind; UNDO interpolates between noising and full reinitialization to trade compute for robustness.
Key results
- Distilling an oracle-matched reference model into a random init student yields relearning speed close to the oracle, while the reference model relearns much faster.
- UNDO with MaxEnt traces a Pareto frontier that approaches data-filtering gold standard robustness on language and arithmetic tasks at lower compute and labeling cost.
- On WMDP, UNDO makes MaxEnt and RMU more resilient to relearning across seven adversarial scenarios, but the UNDO model loses 4.88% MMLU on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill an unlearned model into a randomly initialized student | Unlearn-and-Distill · Language and arithmetic relearning attacks | relearning resistance | Helped |
| Use UNDO after MaxEnt or RMU unlearning | UNDO · WMDP-Bio and WMDP-Cyber | relearning resistance | Helped |
| Increase UNDO mixing coefficient α (ablation) | UNDO · Language and arithmetic relearning | relearning resistance | Helped |
| Increase UNDO mixing coefficient α (ablation) | UNDO · Language and arithmetic distillation | compute required to recover teacher retain performance | Hurt |
| Vary noise scale β (ablation) | UNDO · Language and arithmetic | relearning resistance | No clear effect |
| Use lower β values rather than β = 1.0 (ablation) | UNDO · Language | relearning resistance | Helped |
| Distill an oracle's outputs into a randomly initialized student rather than an oracle-matched reference model (ablation) | Oracle matching · Language and arithmetic forget-domain relearning | relearning speed | Helped |
| Use RMU followed by distillation instead of RMU alone | Unlearn-and-Distill · Arithmetic multiplication and division | relearning resistance | Helped |
| Use UNDO rather than unlearning-only methods | UNDO with MaxEnt · Language and arithmetic under adversarial relearning | forget capability recovery | Helped |
| Apply UNDO to WMDP unlearning | UNDO · WMDP measured on MMLU | MMLU performance: 4.88% lower | Hurt |
| Use RMU instead of MaxEnt as the arithmetic teacher | Unlearn-and-Distill · Arithmetic retain-forget trade-off | retain accuracy: 80% and 1.3% vs. 62% retain and 6.8% forget accuracy | Hurt |
Surprising findings
- Even when a model is trained to match an oracle's outputs with near-zero distillation loss, it relearns the forget capability much faster than the oracle, so behavioral matching is insufficient.
- Lower noise scale beta often worked as well or better than beta=1.0, and simply shrinking parameters (beta=0) was also effective on WMDP.
Limitations
- On WMDP, UNDO costs about 4.88% MMLU accuracy and other methods reach similar trade-offs, so it is not clearly superior there.
- Distillation is considerably more costly than finetuning-based unlearning methods.
- Experiments on the WMDP setting undertrain, distilling on only 0.015% of the pretraining corpus size.
Experimental setup
Datasets: FineWeb-Edu FineWeb2 Korean synthetic arithmetic dataset WMDP MMLU TOFU
Benchmarks: WMDP-Bio WMDP-Cyber MMLU TOFU
Metrics: cross-entropy loss accuracy elicited forget performance after relearning attacks
Baselines: GradDiff MaxEnt RMU SAM RepNoise UNDIAL Data Filtering oracle
Builds on: Gradient Difference unlearning Maximizing Entropy unlearning Representation Misdirection for Unlearning (RMU) UNDIAL Sharpness-Aware Minimization
Closest papers by meaning
- Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.89)
- Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond (ICML 2025 · similarity 0.89)
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.89)
- On Large Language Model Continual Unlearning (ICLR 2025 · similarity 0.88)
- When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.88)
- RULE: Reinforcement UnLEarning Achieves Forget-retain Pareto Optimality (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.