Research map / Adversarial robustness and security / Attacks on text and graphs
Rethinking Adversarial Robustness in the Context of the Right to be Forgotten
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdvUA, an attack that crafts malicious machine unlearning requests (without touching training data) to move victim samples into low-density regions, degrading adversarial robustness of unlearned models and boosting model stealing.
Problem
Machine unlearning ("right to be forgotten") is studied for privacy, but its effect on the adversarial robustness of unlearned models, and whether adversaries can exploit unlearning requests to create adversarial vulnerabilities, is unexplored.
Approach
Select unlearning samples near target victim samples using a local-density metric and a direction-alignment loss with the adversarial gradient, so that unlearning pushes victims into low-density regions where adversarial examples succeed; the attack is orthogonal to existing adversarial attacks and can be combined with them.
Key results
- On adversarially trained ResNet-18 (CIFAR-10), AdvUA with 100 unlearning samples per hard sample reaches about 88% attack success against FGSM and PGD, while random removal performs poorly.
- Transferring FGSM from ResNet-50 to DenseNet-121 on ImageNet, AdvUA reaches 96% success with 30 unlearning samples versus 61% for the baseline.
- In model stealing on IRIS with 100 queries, AdvUA raises extracted-model test accuracy from 45.24% to 85.71% (1.89x).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AdvUA-selected unlearning samples instead of no unlearning | ResNet-50 substitute transferred to VGG-19 · ImageNet | FGSM attack success rate: 63.0% ± 4.7% -> 92.0% ± 2.0% | Helped |
| Use AdvUA-selected unlearning samples instead of no unlearning | ResNet-50 substitute transferred to DenseNet-121 · ImageNet | FGSM attack success rate: 61.0% ± 6.9% -> 96.0% ± 1.6% | Helped |
| Add AdvUA-selected unlearning samples | Query-based model stealing attack · Model stealing, 100 queries | test accuracy: 45.24% ± 6.17% -> 85.71% ± 4.92% | Helped |
| Add AdvUA-selected unlearning samples | Query-based model stealing attack · Model stealing, 200 queries | test accuracy: 56.19% ± 9.85% -> 88.57% ± 2.41% | Helped |
| Add AdvUA-selected unlearning samples | Query-based model stealing attack · Model stealing, 300 queries | test accuracy: 85.56% ± 2.81% -> 90.00% ± 2.85% | Helped |
| Add AdvUA-selected unlearning samples | Query-based model stealing attack · Model stealing, 400 queries | test accuracy: 86.11% ± 5.74% -> 92.22% ± 1.41% | Helped |
| Increase the number of AdvUA-selected unlearning samples | Adversarially trained models · CIFAR-10 and SVHN | attack success rate: approximately 88%; 90% | Helped |
| Select test samples from low-density rather than high-density regions (ablation) | AdvUA · CIFAR-10 | PGD attack success rate | Helped |
| Increase the perturbation bound (ablation) | AdvUA, kNN, and random deletion · CIFAR-10 | CW attack success rate | Mixed |
| Use SISA unlearning instead of first-order unlearning (ablation) | AdvUA · CIFAR-10 | attack success rate: around 66% | Helped |
| Randomly remove training samples | Adversarially trained models · SVHN and CIFAR-10 | robust accuracy | Hurt |
Surprising findings
- Even random removal of training samples substantially amplifies adversarial vulnerability of defended robust models.
- Adversarially robust models are more susceptible to malicious unlearning samples than naturally trained models.
Limitations
- White-box main evaluation; black-box setting relies on substitute models and transferability.
- Algorithms and complexity analysis deferred to an unavailable full version.
- Evaluated only on image classification and small datasets; other domains such as federated learning and graph neural networks left to future work.
Experimental setup
Datasets: CIFAR-10 SVHN ImageNet IRIS
Benchmarks: CIFAR-10 SVHN ImageNet IRIS
Metrics: attack success rate robust accuracy drop ratio attack transferability test accuracy of extracted model
Baselines: random deletion kNN deletion
Builds on: adversarial training (Madry et al.) first-order unlearning (Warnecke et al.) SISA model stealing (Tramèr et al.)
Closest papers by meaning
- Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy (ICLR 2025 · similarity 0.90)
- Adversarial Machine Unlearning (ICLR 2025 · similarity 0.90)
- Flexible, Efficient, and Stable Adversarial Attacks on Machine Unlearning (ICML 2025 · similarity 0.90)
- Backdoor Attacks via Machine Unlearning (AAAI 2024 · similarity 0.89)
- Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond (ICML 2025 · similarity 0.89)
- Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.