Research map / Adversarial robustness and security / Certified and empirical robustness
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace fixed-step PGD with budget-aware APGD | PGD adversarial evaluation · CIFAR-10, l∞-robust models | robust accuracy: 54.84% -> 54.00% | Helped |
| Use the DLR loss instead of cross-entropy | APGD · Deterministic robustness evaluation | robust accuracy: 54.00% -> 48.53% | Helped |
| Combine complementary gradient-based, boundary-based, and score-based attacks | Adversarial robustness evaluation · 49 deterministic models | robust accuracy versus reported evaluations: lower in all but one case; more than 10% in 13 out of 49 cases | Helped |
| Use targeted APGD-DLR instead of untargeted APGD-DLR | APGD-DLR · CIFAR-10, CIFAR-100, and ImageNet | robust accuracy: 35/36 | Helped |
| Use targeted FAB instead of untargeted FAB | FAB · CIFAR-10, CIFAR-100, and ImageNet | robust accuracy: 29/32 | Helped |
| Use targeted attacks instead of untargeted attacks | APGD-DLR and FAB · MNIST | robust accuracy | Hurt |
| Use the DLR loss instead of the CW loss | PGD and APGD attacks · Deterministic models | robust accuracy: up to 21%; never better by more than 5% | Mixed |
| Use APGD instead of fixed-step PGD | Adversarial robustness evaluation · 43 l∞-robust models, CE loss | robust accuracy: 32 of the 43 models | Helped |
| Use APGD instead of fixed-step PGD | Adversarial robustness evaluation · 43 l∞-robust models, CW loss | robust accuracy: 37/43 | Helped |
| Use APGD instead of fixed-step PGD | Adversarial robustness evaluation · 43 l∞-robust models, DLR loss | robust accuracy: 35/43 | Helped |
| Add Square Attack to the ensemble | AutoAttack · Discontinuous or gradient-masked defenses | robust accuracy: 20.44% -> 8.0% | Helped |
| Use a larger Square Attack query budget and more restarts (ablation) | Square Attack · Discontinuous defense, 500 points | robust accuracy: 20.44% -> 8.0% | Helped |
Closest papers by meaning
- Increasing Confidence in Adversarial Robustness Evaluations (NeurIPS 2022 · similarity 0.93)
- Practical Evaluation of Adversarial Robustness via Adaptive Auto Attack (CVPR 2022 · similarity 0.92)
- Accurate, reliable and fast robustness evaluation (NeurIPS 2019 · similarity 0.92)
- Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples (NeurIPS 2022 · similarity 0.92)
- Benchmarking Adversarial Robustness on Image Classification (CVPR 2020 · similarity 0.92)
- Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attacks (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.