Research map / Adversarial robustness and security / Attacks on text and graphs
From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that approximate example-level unlearning in vision classifiers is vulnerable to relearning attacks (fine-tuning on the retain set restores forget accuracy), links resistance to weight-space L2 distance and linear mode connectivity, and proposes weight-space regularized unlearning methods (Weight Distortion, Weight Dist Reg, CBFT) that are more tamper-resistant.
Problem
Approximate unlearning methods for LLMs and classifiers appear to remove forget-set accuracy but can be reversed by fine-tuning (relearning/tampering attacks), even on only retain data. It is unclear what makes unlearned models resistant to such attacks in a controlled setting.
Approach
Study relearning attacks in a controlled example-level unlearning setting on vision classifiers where retrain-from-scratch is a clear gold standard; find that resistance correlates with weight-space L2 distance from the pretrained model and loss barriers under linear mode connectivity, and use these insights to design regularizers that push the unlearned model far from the pretrained model in weig
Key results
- Forget-set accuracy for several unlearning methods (e.g., Circuit Breakers, SCRUB, Random Relabeling) recovers from about 50% post-unlearning to nearly 100% after fine-tuning on the retain set only, whereas retrain-from-scratch stays at about 50%.
- Methods with larger weight-space L2 distance from the pretrained model (Catastrophic Forgetting, L1-Sparse, Weight Distortion, Weight Dist Reg) show higher tamper resistance; Weight Dist Reg and Weight Distortion are significantly more resistant than prior methods.
- TAR provides little resistance to relearning in this vision setting, while CBFT and Weight Dist Reg improve resistance when applied as a second phase on an initial safeguard.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune an unlearned model on the retain set only | SCRUB and other approximate unlearning methods · CIFAR-10, atypical forget examples | forget set accuracy: near 50% -> nearly 100% | Hurt |
| Fine-tune the retrained-from-scratch model on the retain set only | Retrain from Scratch · CIFAR-10, atypical forget examples | forget set accuracy: 50% -> 50% | No clear effect |
| Increase distance between pretrained and unlearned weights | Weight Dist Reg · Atypical forget examples across evaluated settings | forget set accuracy after relearning | Helped |
| Add Gaussian noise to weights before retain-set fine-tuning | Weight Distortion · CIFAR-10, atypical forget examples; 0 Relearn | forget set accuracy: 99.99% -> 61.07% | Helped |
| Use weight distortion rather than catastrophic forgetting for 1-epoch unlearning | Weight Distortion · CIFAR-10, atypical forget examples; 0 Relearn | forget set accuracy: 99.99% -> 61.07% | Helped |
| Use Weight Dist Reg instead of SCRUB | Imagenette · Imagenette, 0 Relearn | forget set accuracy: 93.40% -> 73.87% | Helped |
| Use CBFT rather than Weight Dist Reg or Weight Distortion | CBFT · CIFAR-10, atypical forget examples | tamper resistance | Hurt |
| Replace the retain set with held-out test examples during relearning | Unlearned models · CIFAR-10, atypical forget examples | forget set accuracy | Hurt |
| Use atypical rather than typical forget examples | Unlearning and relearning methods · CIFAR-10 | forget set accuracy after relearning | Mixed |
| Apply Weight Dist Reg as a second phase to an initial safeguard | Two-phase unlearning methods · CIFAR-10, atypical forget examples | tamper resistance | Helped |
| Apply CBFT as a second phase to an initial safeguard | Two-phase unlearning methods · CIFAR-10, atypical forget examples | tamper resistance | Helped |
| Increase unlearning duration for Catastrophic Forgetting from 1 to 100 epochs (ablation) | Catastrophic Forgetting · CIFAR-10, atypical forget examples; 0 Relearn | forget set accuracy: 99.99% -> 66.14% | Helped |
Surprising findings
- Fine-tuning on the retain set alone (zero forget examples) restores forget-set accuracy to nearly 100% for many unlearning methods.
- TAR, which was designed for tamper resistance in LLMs, provides essentially no resistance in this vision setting.
- Vision classifiers are not vulnerable to quantization attacks until near 8 bits, unlike language models.
Limitations
- Experiments are limited to small ResNet models (ResNet-18/34) and small-resolution CIFAR datasets, with only limited higher-resolution (Imagenette) validation.
- Tamper resistance trades off against test accuracy; the most robust methods have the lowest test accuracy.
- Hyperparameters transferred directly to ResNet-34 made Weight Dist Reg susceptible to relearning, indicating sensitivity to hyperparameters.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Imagenette CIFAR-10-C
Benchmarks: CIFAR-10 CIFAR-100 Imagenette
Metrics: forget set accuracy test accuracy relearning accuracy L2 distance linear mode connectivity loss barrier membership inference attack accuracy
Baselines: SCRUB Circuit Breakers NegGrad+ Catastrophic Forgetting L1-Sparse Selective Synaptic Dampening Random Relabeling TAR Retrain from scratch
Builds on: Tampering Attack Resistance (TAR) Circuit Breakers SCRUB Catastrophic Forgetting Selective Synaptic Dampening
Closest papers by meaning
- Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond (ICML 2025 · similarity 0.91)
- The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples (NeurIPS 2025 · similarity 0.91)
- Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning (ICLR 2025 · similarity 0.90)
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.90)
- Not All Wrong is Bad: Using Adversarial Examples for Unlearning (ICML 2025 · similarity 0.90)
- Learning to Unlearn: Instance-Wise Unlearning for Pre-trained Classifiers (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.