Research map / Adversarial robustness and security / Certified and empirical robustness
Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves on a patch-structured robust/non-robust feature data model that standard training of a two-layer smoothed-ReLU CNN learns non-robust features, while gradient-based adversarial training (one-step attack) learns robust features and suppresses non-robust ones, with small MNIST/CIFAR10/SVHN experiments.
Problem
Explaining why adversarial examples exist in networks trained by standard training and how adversarial training improves robustness, from a network-optimization (feature learning) perspective, under data more realistic than Gaussian-mixture or multi-cluster models.
Approach
A patch-structured data model with robust (strong, sparse) and non-robust (weak, dense) features, trained with a two-layer smoothed-ReLU CNN; the feature learning dynamics show standard GD learns non-robust features first, while one-step gradient-ascent adversarial training shifts learning toward robust features.
Key results
- On CIFAR10, standard training yields 95.74% standard accuracy but 0.00% robust accuracy, while adversarial training yields 86.28% standard and 45.13% robust accuracy.
- On MNIST, adversarial training raises robust test accuracy from 0.04% (standard training) to 94.43%, while reducing non-robust feature learning from 88.11 to 39.49.
- Targeted NRF-PGD attacks on the standard-trained CIFAR10 network achieve 54-80% success rates, versus 1-12% for RF-PGD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adversarial training instead of standard training | ResNet18 · MNIST | robust test accuracy: 0.04 -> 94.43 | Helped |
| Use adversarial training instead of standard training | WideResNet-34-10 · CIFAR10 | robust test accuracy: 0.00 -> 45.13 | Helped |
| Use adversarial training instead of standard training | WideResNet-34-10 · SVHN | robust test accuracy: 0.16 -> 58.55 | Helped |
| Use adversarial training instead of standard training | ResNet18 · MNIST | standard test accuracy: 99.56 -> 99.41 | Hurt |
| Use adversarial training instead of standard training | WideResNet-34-10 · CIFAR10 | standard test accuracy: 95.74 -> 86.28 | Hurt |
| Use adversarial training instead of standard training | WideResNet-34-10 · SVHN | standard test accuracy: 96.84 -> 91.99 | Hurt |
| Use adversarial training instead of standard training | ResNet18 · MNIST | robust feature learning accuracy: 44.61 -> 69.19 | Helped |
| Use adversarial training instead of standard training | WideResNet-34-10 · CIFAR10 | robust feature learning accuracy: 61.26 -> 67.88 | Helped |
| Use adversarial training instead of standard training | WideResNet-34-10 · SVHN | robust feature learning accuracy: 65.38 -> 77.04 | Helped |
| Use adversarial training instead of standard training | WideResNet-34-10 · CIFAR10 | non-robust feature learning accuracy: 91.79 -> 10.88 | Hurt |
| Use adversarial training instead of standard training | WideResNet-34-10 · SVHN | non-robust feature learning accuracy: 73.91 -> 35.50 | Hurt |
| Use adversarial training instead of standard training | ResNet18 · MNIST | non-robust feature learning accuracy: 88.11 -> 39.49 | Hurt |
Surprising findings
- Standard training's non-robust feature learning is shown to dominate even though non-robust features are dense and weak, implying adversarial examples arise from learned non-robust features.
Limitations
- Analysis is for two-layer CNNs and one-step gradient-ascent adversarial training rather than multi-step PGD.
- Theory relies on noiseless population risk simplifications and strong data assumptions (alpha >> epsilon > beta, orthogonal features).
- Extension to deep neural networks is left as future work.
Experimental setup
Datasets: MNIST CIFAR10 SVHN synthetic patch-structured data
Benchmarks: MNIST CIFAR10 SVHN
Metrics: standard test accuracy robust test accuracy feature learning accuracy targeted attack success rate
Baselines: standard training adversarial training
Builds on: feature purification robust and non-robust features decomposition patch-structured data (multi-view ensembles) tensor power method
Closest papers by meaning
- Improving Adversarial Robustness Requires Revisiting Misclassified Examples (ICLR 2020 · similarity 0.94)
- Robust Models are less Over-Confident (NeurIPS 2022 · similarity 0.94)
- Attribute-Guided Adversarial Training for Robustness to Natural Perturbations (AAAI 2021 · similarity 0.92)
- Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samples (ICML 2023 · similarity 0.92)
- On Adversarial Training without Perturbing all Examples (ICLR 2024 · similarity 0.92)
- Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.