Research map / Adversarial robustness and security / Certified and empirical robustness
Theoretical Understanding of Learning from Adversarial Perturbations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives a theoretical justification for learning from adversarial perturbations using a one-hidden-layer leaky-ReLU network trained by gradient flow on near-orthogonal samples, showing perturbations (including sparse L0 ones) carry class features and the learned decision boundary matches the standard one except in specific regions.
Problem
Why networks trained only on mislabeled adversarial perturbations generalize to correctly labeled test data, and whether adversarial perturbations contain class features; prior work was empirical only.
Approach
Uses the implicit-bias result for one-hidden-layer networks on orthogonal data to derive closed-form decision boundaries when training on geometry-inspired adversarial perturbations of natural samples or of uniform noise, showing the perturbation is a weighted sum of training samples and the learned boundary stays aligned with the standard one.
Key results
- On MNIST, learning from L2 perturbations on natural samples with random targets reaches 92.9% test accuracy, and L0 random-target learning reaches 28.4%.
- On CIFAR-10, learning from L0 perturbations on natural samples with random targets reaches 54.9% test accuracy, while noise-data scenarios give about 10% (near chance).
- Decision boundaries learned from adversarial perturbations align with the standard boundary as input dimension and number of adversarial samples grow, whereas pure noise boundaries deviate markedly.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use random rather than deterministic target labels for natural-sample L0 perturbations | Learning from adversarial perturbations · MNIST | accuracy: 0.71 -> 28.4 | Helped |
| Use random rather than deterministic target labels for natural-sample L2 perturbations | Learning from adversarial perturbations · MNIST | accuracy: 38.4 -> 92.9 | Helped |
| Use random rather than deterministic target labels for natural-sample L∞ perturbations | Learning from adversarial perturbations · MNIST | accuracy: 10.3 -> 89.1 | Helped |
| Use random rather than deterministic target labels for natural-sample L0 perturbations | Learning from adversarial perturbations · Fashion-MNIST | accuracy: 1.31 -> 10.5 | Helped |
| Use random rather than deterministic target labels for natural-sample L2 perturbations | Learning from adversarial perturbations · Fashion-MNIST | accuracy: 25.1 -> 54.8 | Helped |
| Use random rather than deterministic target labels for natural-sample L∞ perturbations | Learning from adversarial perturbations · Fashion-MNIST | accuracy: 22.9 -> 61.4 | Helped |
| Use random rather than deterministic target labels for natural-sample L0 perturbations | Learning from adversarial perturbations · CIFAR-10 | accuracy: 16.8 -> 54.9 | Helped |
| Use random rather than deterministic target labels for natural-sample L2 perturbations | Learning from adversarial perturbations · CIFAR-10 | accuracy: 42.8 -> 77.1 | Helped |
| Use random rather than deterministic target labels for natural-sample L∞ perturbations | Learning from adversarial perturbations · CIFAR-10 | accuracy: 51.5 -> 77.2 | Helped |
| Train on noise data rather than natural samples with L0 perturbations | Learning from adversarial perturbations · Fashion-MNIST | accuracy: 10.5 -> 26.2 | Helped |
| Train on noise data rather than natural samples with L0 perturbations | Learning from adversarial perturbations · CIFAR-10 | accuracy: 54.9 -> 9.87 | Hurt |
| Train on noise data rather than natural samples with L2 perturbations | Learning from adversarial perturbations · CIFAR-10 | accuracy: 77.1 -> 10.2 | Hurt |
Surprising findings
- Sparse L0 perturbations (a few pixels) suffice for generalization on MNIST and CIFAR-10 natural-sample settings.
- Classifiers trained only on perturbed uniform noise classify natural training samples almost perfectly on artificial data, despite never seeing natural data.
Limitations
- Analysis restricted to one-hidden-layer networks trained on mutually orthogonal samples.
- Orthogonality restricts the perturbation budget to about O(sqrt(d/N)), whereas practical epsilon is about O(sqrt(d)).
- CIFAR-10 noise-data scenarios fail, attributed to domain gap between noise and natural images.
Experimental setup
Datasets: MNIST Fashion-MNIST CIFAR-10 uniform noise artificial dataset
Benchmarks: MNIST Fashion-MNIST CIFAR-10
Metrics: test accuracy decision boundary alignment
Baselines: standard training pure uniform noise training
Builds on: learning from adversarial perturbations (Ilyas et al. 2019) implicit bias of gradient flow for one-hidden-layer networks (Kou et al.)
Closest papers by meaning
- Wide Two-Layer Networks can Learn from Adversarial Perturbations (NeurIPS 2024 · similarity 0.96)
- Lower bounds on the robustness to adversarial perturbations (NeurIPS 2017 · similarity 0.90)
- Defense Against Universal Adversarial Perturbations (CVPR 2018 · similarity 0.89)
- Identifying and Understanding Cross-Class Features in Adversarial Training (ICML 2025 · similarity 0.88)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.87)
- Contrasting Adversarial Perturbations: The Space of Harmless Perturbations (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.