Research map / Adversarial robustness and security / Certified and empirical robustness
On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows adversarial overfitting stems from fitting hard adversarial training instances, with linear and nonlinear theory, and mitigates it via adaptive reweighting and adaptive targets in fast AT and adversarial fine-tuning.
Problem
Adversarial training suffers severe robust overfitting in the late training phase, and its cause is poorly understood; the paper asks how training-instance difficulty drives this.
Approach
Defines a relative instance-difficulty metric based on the percentile of each instance's average loss over training epochs; shows that fitting hard adversarial instances causes adversarial overfitting, proves for logistic regression on a GMM that harder instances yield larger robust test error (gap grows with epsilon) and lower-bounds the Lipschitz constant for nonlinear models, then uses adaptive
Key results
- Training PGD adversarial training on the 10k hardest CIFAR10 instances yields near-trivial test accuracy, while the 10k easiest instances show no test degradation during training.
- Adaptive-target fast adversarial training (ATTA variant) reaches 51.17% AutoAttack accuracy on WRN34 with 1/4 of the training cost of the strongest baseline (51.12% at 7 complexity).
- Reweighting plus KL regularization in adversarial fine-tuning raises WRN34 CIFAR10 AutoAttack accuracy from 55.49% (vanilla, 5 epochs) to 56.99%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an adaptive target instead of the standard ATTA target | Fast adversarial training on WRN34 · CIFAR10 | AutoAttack robust accuracy (%): 44.48 -> 51.17 | Helped |
| Downweight hard instances during fast adversarial training | ATTA on WRN34 · CIFAR10 | AutoAttack robust accuracy (%): 44.48 -> 46.15 | Helped |
| Use adaptive targets rather than one-hot targets (ablation) | Fast adversarial training · CIFAR10 | robust test error | Helped |
| Add reweighting to ATTA (ablation) | Fast adversarial training · CIFAR10 | robust test error | Helped |
| Reweight fine-tuning instances (ablation) | Adversarial fine-tuning · CIFAR10, WRN34, 1 epoch | AutoAttack robust accuracy (%): 54.11 -> 54.69 | Helped |
| Add clean-to-adversarial KL regularization (ablation) | Adversarial fine-tuning · SVHN, RN18, 1 epoch | AutoAttack robust accuracy (%): 70.81 -> 72.29 | Helped |
| Combine reweighting and KL regularization (ablation) | Adversarial fine-tuning · CIFAR10, WRN34, 5 epochs | AutoAttack robust accuracy (%): 55.49 -> 56.99 | Helped |
| Combine reweighting and KL regularization (ablation) | Adversarial fine-tuning · SVHN, RN18, 5 epochs | AutoAttack robust accuracy (%): 72.18 -> 73.35 | Helped |
| Train on hard rather than easy instances | PGD adversarial training · CIFAR10, 10,000-instance subset | robust test accuracy | Hurt |
| Train on hard rather than easy instances | PGD adversarial training · CIFAR10, l∞ adversarial budget | Lipschitz upper bound (× 10^4): 14.54 \pm 0.02 -> 567.24 \pm 0.59 | Hurt |
| Increase the l∞ adversarial budget | PGD adversarial training on Easy10K · CIFAR10 | Lipschitz upper bound (× 10^4): 5.91 \pm 0.00 -> 14.54 \pm 0.02 | Hurt |
| Increase the l2 adversarial budget | PGD adversarial training on Easy10K · CIFAR10 | Lipschitz upper bound (× 10^4): 3.34 \pm 0.01 -> 3.91 \pm 0.00 | Hurt |
Surprising findings
- Training on the hardest instances does not produce adversarial overfitting-free models even with longer training or different optimizers; test accuracy stays near trivial.
- The instance-wise reweighting method that upweights hard adversarial instances (IAT-like reweighting from prior work) is shown to be vulnerable to adaptive attacks, contrary to its goal.
Limitations
- Theoretical analysis for nonlinear models relies on a Lipschitz lower bound and assumptions (isoperimetry, bounded parameter space), and the Lipschitz constant is only estimated numerically.
- Fast adversarial training and fine-tuning experiments are preliminary and limited to CIFAR10 and SVHN.
Experimental setup
Datasets: CIFAR10 SVHN
Benchmarks: CIFAR10 SVHN
Metrics: robust test error robust accuracy AutoAttack accuracy Lipschitz constant upper bound
Baselines: vanilla adversarial training ATTA instance-wise adversarial training self-adaptive training early stopping
Builds on: PGD adversarial training ATTA self-adaptive training instance-wise adversarial training
Closest papers by meaning
- Clarifying the Behavior and the Difficulty of Adversarial Training (AAAI 2024 · similarity 0.86)
- A Closer Look at Curriculum Adversarial Training: From an Online Perspective (AAAI 2024 · similarity 0.84)
- Benign Overfitting in Adversarial Training of Neural Networks (ICML 2024 · similarity 0.84)
- Revisiting Adversarial Training Under Long-Tailed Distributions (CVPR 2024 · similarity 0.84)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.84)
- Adversarial Training Should Be Cast as a Non-Zero-Sum Game (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.