Research map / Label-efficient and robust learning / Recognition and long-tail
How Far Are We from True Unlearnability?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that existing unlearnable examples fail to stay unlearnable in multi-task (Taskonomy) settings, proposes Sharpness-Aware Learnability (SAL) and Unlearnable Distance (UD) based on loss-landscape sharpness, and benchmarks mainstream unlearnable methods with UD.
Problem
Existing unlearnable examples (UEs) are designed against single-task classification, and it is unclear whether they remain unlearnable across multiple tasks such as segmentation and depth estimation; existing evaluation relies on test accuracy, which does not explain why UEs work.
Approach
Explain unlearnability via the loss landscape and parameter sharpness: UEs reduce the sharpness-aware learnability (SAL) of parameters, so the fraction of parameters with high SAL (split by a K-means threshold on clean training) gives an Unlearnable Distance measuring how far a UE set is from truly unlearnable.
Key results
- On ImageNet-100 with ResNet-18, EM achieves the lowest UD (0.011), while on CIFAR-10 OPS (0.157) and EM (0.187) are best and TAP has UD 1.639, above 1.
- On CIFAR-10 ResNet-18, UEs reduce test accuracy to 10-30% (e.g., OPS 11.98%, EM 26.52%) versus 94.11% for clean training.
- Defenses such as JPEG compression raise UD for EM and OPS, while TAP's UD drops; ViT yields the highest UD for EM (1.573), suggesting stronger models make unlearnability harder to maintain.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EM perturbations instead of clean training | ResNet-18 · Taskonomy multi-task training | task performance | No clear effect |
| Use existing unlearnable examples instead of clean data | ResNet multi-task model · Taskonomy | task performance | No clear effect |
| Use EM perturbations generated for one task | Taskonomy multi-task model · cross-task training | task performance | No clear effect |
| Use EM instead of clean data | ResNet-18 · CIFAR-10 | UD: 0.187 | Helped |
| Use OPS instead of clean data | ResNet-18 · CIFAR-10 | UD: 0.157 | Helped |
| Use TAP instead of clean data | ResNet-18 · CIFAR-10 | UD: 1.639 | Hurt |
| Use EM instead of clean data | ResNet-18 · CIFAR-100 | UD: 0.111 | Helped |
| Use OPS instead of clean data | ResNet-18 · CIFAR-100 | UD: 0.009 | Helped |
| Use TAP instead of clean data | ResNet-18 · CIFAR-100 | UD: 1.213 | Hurt |
| Use EM instead of clean data | ResNet-18 · ImageNet-100 | UD: 0.011 | Helped |
| Use OPS instead of clean data | ResNet-18 · ImageNet-100 | UD: 1.430 | Hurt |
| Use TAP instead of clean data | ResNet-18 · ImageNet-100 | UD: 2.269 | Hurt |
Surprising findings
- Existing UEs such as EM fail to exhibit unlearnability in multi-task training on Taskonomy, with minimal impact on task metrics.
- TAP, an adversarial-example attack, shows UD above 1 (more learnable parameters than clean), so UD classifies it as not a true unlearnable example despite low test accuracy.
Limitations
- SAL computation is time-consuming, so multi-task analysis uses heat maps and the UD is only labeled rather than systematically computed for multi-task models.
- The threshold relies on K-means on clean-model SAL, and the paper notes shortcut learning does not fully explain unlearnability.
- Multi-task analysis uses a tiny Taskonomy split and a single ResNet backbone.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-100 Taskonomy MNIST make_classification (scikit-learn)
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-100 Taskonomy
Metrics: test accuracy Unlearnable Distance (UD) number of learnable parameters (#LP)
Baselines: EM REM DC TAP LSP OPS
Builds on: Sharpness-Aware Minimization Error-Minimizing noise (EM) OPS K-Means
Closest papers by meaning
- A3: Few-shot Prompt Learning of Unlearnable Examples with Cross-Modal Adversarial Feature Alignment (CVPR 2025 · similarity 0.87)
- One for All: A Universal Generator for Concept Unlearnability via Multi-Modal Alignment (ICML 2024 · similarity 0.87)
- Ungeneralizable Examples (CVPR 2024 · similarity 0.87)
- On Large Language Model Continual Unlearning (ICLR 2025 · similarity 0.87)
- Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models (ICLR 2024 · similarity 0.86)
- Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.