Research map / Adversarial robustness and security / Certified and empirical robustness
Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives the first scaling laws for CIFAR10 adversarial training that model synthetic data quality (FID), uses them to train compute-efficient WideResNets reaching 73.71% AutoAttack accuracy, and runs a small human study showing ~10% of AutoAttack examples are invalid for human labels.
Problem
Adversarial robustness on CIFAR10 under l-infinity (8/255) attacks remains far from solved (~71% SOTA on RobustBench) despite near-100% clean accuracy; it is unclear whether scale or the attack formulation is the bottleneck.
Approach
Fit adversarial-training scaling laws over model size, dataset size, and synthetic data FID (three approaches), use them to pick compute-optimal WideResNet configurations, extrapolate a robustness asymptote near 90%, and test it with a human study showing that l-infinity attacks create invalid images that humans also misclassify.
Key results
- New SOTA of 73.71% AutoAttack accuracy (93.68% clean) by training WideResNet-94-16 on 500M samples, beating the prior 71.07% level.
- A compute-efficient WideResNet-82-8 (79M params) reaches 71.59% AutoAttack, matching prior SOTA robustness with about 20% fewer training FLOPs and 3x fewer parameters.
- Scaling laws predict robustness plateaus near 90%; human study estimates average human accuracy on all adversarial images of about 90.46%, with 727 of 2629 misclassified-by-model images deemed invalid.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use compute-efficient model and dataset sizing | CIFAR10 adversarial training · CIFAR10 test, AutoAttack | accuracy: +3% AutoAttack accuracy; 73.71% vs. prior SOTA | Helped |
| Use a compute-optimal smaller model and more data at the prior SOTA compute budget | CIFAR10 adversarial training · CIFAR10 test, AutoAttack | accuracy: +1% AutoAttack; 71.59% vs. prior SOTA | Helped |
| Train entirely on synthetic data | CIFAR10 adversarial training · CIFAR10 test, AutoAttack | accuracy: 71.7% vs. prior SOTA 71.07% | Helped |
| Increase synthetic dataset quality | CIFAR10 adversarial training · CIFAR10 | adversarial loss | Helped |
| Scale model and dataset sizes at roughly the same rate on high-quality synthetic data | CIFAR10 adversarial training · CIFAR10 | scaling-law exponent: a=0.56, b=0.43 for PFGM++; a=0.48, b=0.52 for Approach 2 | Mixed |
| Exclude invalid adversarial images from the benchmark | SOTA NN · CIFAR10 test, AutoAttack valid data | accuracy: 73.71% -> 79.49% | Helped |
| Exclude invalid adversarial images | Estimated average human performance · CIFAR10 AutoAttack | estimated average human accuracy: 90.46% -> 96.46% | Helped |
| Use a larger model at a fixed compute budget instead of the compute-optimal model size | CIFAR10 adversarial training · CIFAR10 test, AutoAttack | accuracy | Hurt |
| Infrequently resample training examples instead of reloading the sampler state (ablation) | CIFAR10 scaling-law training runs · CIFAR10 | performance | No clear effect |
| Train with 500M synthetic samples and mixed CIFAR10 data | WideResNet-94-16 · CIFAR10 test, AutoAttack | accuracy: 70.69% -> 73.71% | Helped |
Surprising findings
- Roughly 28% (727/2629) of AutoAttack images the SOTA model misclassified were judged invalid by humans, indicating l-infinity attacks can create images that no longer fit their label.
- Prior SOTA was trained with too large a model for its FLOPs budget; training smaller models on more data gave better robustness.
Limitations
- Human study limited to three research-team members plus GPT-4 on 2629 images.
- Findings tested only on CIFAR10 with WideResNets; generalization to ImageNet or ViTs untested.
- Scaling-law fits rely on a restricted set of runs (dropped small-model/large-data runs).
Experimental setup
Datasets: CIFAR10 synthetic CIFAR10 from EDM synthetic CIFAR10 from PFGM++ synthetic CIFAR10 from Discriminator Guidance
Benchmarks: CIFAR10 Linf AutoAttack (8/255) CIFAR10 PGD-40 CW loss test accuracy
Metrics: AutoAttack accuracy clean accuracy adversarial loss FID human classification accuracy predicted loss
Baselines: Gowal et al. 2021 WideResNet-70-16 adversarial training (prior SOTA) RobustBench CIFAR10 Linf leaderboard entries
Builds on: Chinchilla scaling laws TRADES RobustBench AutoAttack PFGM++
Closest papers by meaning
- Fantastic Robustness Measures: The Secrets of Robust Generalization (NeurIPS 2023 · similarity 0.90)
- Adversarial Robustness without Adversarial Training: A Teacher-Guided Curriculum Learning Approach (NeurIPS 2021 · similarity 0.90)
- RobustBench: a standardized adversarial robustness benchmark (NeurIPS 2021 · similarity 0.90)
- ROME is Forged in Adversity: Robust Distilled Datasets via Information Bottleneck (ICML 2025 · similarity 0.88)
- Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attacks (CVPR 2025 · similarity 0.88)
- GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.