Research map / Adversarial robustness and security / Certified and empirical robustness
Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models
Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, Yupeng Gao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| higher classification accuracy (lower error) | ImageNet classifiers · ImageNet, I-FGSM and C&W | adversarial distortion: reduce error by half -> l_inf distortion falls about 0.02 | Hurt |
| move from AlexNet to most accurate NASNet | ImageNet classifiers · ImageNet, I-FGSM and C&W | robustness: 56.9% -> 82.7% top-1 | Hurt |
| increase network depth within a family | DenseNet, ResNet, Inception · ImageNet, I-FGSM | l_inf distortion | Improved |
| increase model size across families | ImageNet classifiers · ImageNet | robustness: 60 million vs 1.5 million parameters | No clear effect |
| increase FGSM epsilon (targeted) | FGSM · ImageNet, targeted | attack success rate: below 3% | Hurt |
| use iterative instead of single-step FGSM | I-FGSM · ImageNet, targeted | attack success rate: only 10 iterations | Improved |
| increase confidence kappa | C&W and EAD-L1 · ImageNet | attack success rate: kappa 0 near 100% | Hurt |
| increase kappa (untargeted transfer) | C&W and EAD-L1 · ImageNet transfer attack | transferability: kappa 40 | Improved |
| use VGG 16/19 as source model | Transfer attacks · ImageNet, 18 models | transferability | Improved |
| use fewer I-FGSM iterations | I-FGSM · ImageNet transfer attack | transferability | Improved |
| use C&W instead of FGSM for transfer | Transfer attacks · ImageNet untargeted | transferability: FGSM >= I-FGSM >= EAD-L1 >= C&W | Hurt |
| restrict to 327 images correct for all models | Robustness analysis · ImageNet (327 images) | robustness trend: 1,000 -> 327 images | No clear effect |
Closest papers by meaning
- A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking (IJCV 2024 · similarity 0.94)
- Robust Models are less Over-Confident (NeurIPS 2022 · similarity 0.92)
- Benchmarking Adversarial Robustness on Image Classification (CVPR 2020 · similarity 0.92)
- Do Adversarially Robust ImageNet Models Transfer Better? (NeurIPS 2020 · similarity 0.92)
- Neural Architecture Design and Robustness: A Dataset (ICLR 2023 · similarity 0.91)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
