Research map / Adversarial robustness and security / Black-box and transfer attacks
Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that adversarially perturbed unlearning requests for data not in the training set can drive gradient-ascent-based unlearning to near-chance accuracy, using white-box and black-box attacks on CIFAR-10 and ImageNet, and evaluates request-verification defenses.
Problem
Machine unlearning protocols assume the forget set is a subset of the training data; when this assumption is violated, an attacker can submit crafted unlearning requests that degrade model accuracy on retained data. Current verification mechanisms for detecting such requests are unclear.
Approach
Craft adversarial forget sets (visually similar to valid examples, or even random pixels) by differentiating through the unlearning update (gradient-through-gradient) in white-box settings, and by zeroth-order gradient estimation in black-box settings, so that gradient-ascent-style unlearning on these requests destroys retain accuracy.
Key results
- White-box attack reduces CIFAR-10 test accuracy to 3.6% and ImageNet to 0.4%, with retain accuracy drops up to about 96%.
- Black-box attack reduces CIFAR-10 test accuracy to 8.5% and ImageNet to 1.3%, with retain drops up to 89.6% on CIFAR-10 and 80.3% on ImageNet.
- Stealthy adversarial forget sets (l2 projection) push AUROC of image hashing and LPIPS detectors well below 0.5, so most verification schemes fail to separate them from benign JPEG-recompressed requests.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Submit adversarial unlearning requests not present in training data using the white-box attack | GA-family unlearning on image classifiers · CIFAR-10 and ImageNet test sets | accuracy: 3.6\% on CIFAR-10; 0.4\% on ImageNet after the attack | Hurt |
| Submit adversarial unlearning requests not present in training data using the black-box attack | GA-family unlearning on image classifiers · CIFAR-10 and ImageNet test sets | accuracy: 8.5\% on CIFAR-10; 1.3\% on ImageNet after the attack | Hurt |
| Replace white-box gradients with black-box loss-query gradient estimates | Adversarial unlearning attack · CIFAR-10 and ImageNet retain sets | accuracy drop: up to an 89.6\% drop on CIFAR-10 and 80.3\% on ImageNet | Mixed |
| Increase the forget-set size | Black-box adversarial attack · CIFAR-10 and ImageNet | accuracy drop | Hurt |
| Transfer adversarial forget sets generated on a shadow model to the target model | CIFAR-10 unlearning · CIFAR-10 retain set | accuracy drop: 68.04 for forget size 10; 3.91 for forget size 100 | Helped |
| Attack GA_GDR or GA_KLR rather than GA | White-box adversarial unlearning attack · CIFAR-10 and ImageNet retain sets | accuracy drop: CIFAR-10: 91.91, 92.66, 93.55, 94.01; ImageNet: 93.95, 94.99, 96.62, 96.79 | Helped |
| Constrain adversarial image changes with an ℓ2 norm limit | Adversarial request detection · CIFAR-10 requests | AUROC: AUROC values often drop well below 0.5 | Hurt |
| Use LPIPS detection under a stealthy attack instead of an unbounded attack | LPIPS request verification · CIFAR-10 requests | AUROC | Hurt |
| Initialize the black-box attack with random pixels rather than training examples | CIFAR-10 black-box attack · CIFAR-10 retain set, forget size 10 | accuracy drop: 67.14 | Helped |
| Initialize the black-box attack with CIFAR-100 examples rather than training examples | CIFAR-10 black-box attack · CIFAR-10 retain set, forget size 10 | accuracy drop: 30.69 | Helped |
| Select valid training examples strategically for the forget set rather than choosing a normal forget set | GA unlearning with similarity-search protocol · CIFAR-10 retain set | retain error: 3.8\% -> 30.9\% | Hurt |
Surprising findings
- Adversarial forget sets initialized from random pixels or CIFAR-100 images still cause large retain accuracy drops, so the attacker need not own training data.
- LPIPS is highly effective against non-stealthy attacks but becomes the weakest detector once stealthy tactics are applied.
Limitations
- Evaluation limited to image classification with gradient-ascent-family unlearning methods.
- Theoretical result is for linear models and does not capture neural network non-linearity.
- Black-box attack effectiveness degrades with larger forget sets in high-dimensional settings.
Experimental setup
Datasets: CIFAR-10 ImageNet CIFAR-100
Benchmarks: CIFAR-10 ImageNet
Metrics: accuracy drop on retain set accuracy drop on holdout set AUROC retain error
Baselines: GA GA_GDR GA_KLR
Builds on: MAML gradient ascent unlearning NeurIPS 2023 Machine Unlearning Challenge
Closest papers by meaning
- Not All Wrong is Bad: Using Adversarial Examples for Unlearning (ICML 2025 · similarity 0.92)
- Adversarial Machine Unlearning (ICLR 2025 · similarity 0.91)
- Rethinking Adversarial Robustness in the Context of the Right to be Forgotten (ICML 2024 · similarity 0.90)
- Machine Unlearning Fails to Remove Data Poisoning Attacks (ICLR 2025 · similarity 0.90)
- The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples (NeurIPS 2025 · similarity 0.90)
- Learning to Unlearn: Instance-Wise Unlearning for Pre-trained Classifiers (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.