Research map / Adversarial robustness and security / Certified and empirical robustness
Increasing Confidence in Adversarial Robustness Evaluations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an active binarization test to check whether the evaluation attack finds guaranteed adversarial examples | Adversarial robustness evaluations · 13 previously published defenses | evaluations flagged as insufficient: eleven out of thirteen | Helped |
| Freeze batch-normalization statistics during evaluation and increase PGD steps from 20 to 75 | Defense with batch-normalization evaluation bug · CIFAR-10 test set | binarization test score: 0.04 -> 1.0 | Helped |
| Freeze batch-normalization statistics during evaluation and increase PGD steps from 20 to 75 | Defense with batch-normalization evaluation bug · CIFAR-10 test set | robust accuracy: 60.15\,\% -> \leq 1\,\% | Hurt |
| Make the attack adaptive to the detector using feature matching | Detector-based defense · CIFAR-10 test set | binarization test score: 0.26 and 0.63 -> 0.99 | Helped |
| Make the attack adaptive to the detector using feature matching | Detector-based defense · CIFAR-10 test set | robust accuracy: 99\,\% -> below 12\,\% | Hurt |
| Use FAB instead of AutoPGD | Defense evaluated with PGD, AutoPGD, and FAB · CIFAR-10 test set | robust accuracy: 8.16\,\% -> 0.71\,\% | Hurt |
| Use FAB instead of original PGD | Defense evaluated with PGD, AutoPGD, and FAB · CIFAR-10 test set | robust accuracy: 32.32\,\% -> 0.71\,\% | Hurt |
| Use the ground-truth label rather than the classifier's predicted label as the attack target | Defense evaluation with an attack targeting predicted labels · CIFAR-10 test set | robust accuracy | Hurt |
Closest papers by meaning
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks (ICML 2020 · similarity 0.93)
- Accurate, reliable and fast robustness evaluation (NeurIPS 2019 · similarity 0.92)
- Benchmarking Adversarial Robustness on Image Classification (CVPR 2020 · similarity 0.92)
- Towards Deep Learning Models Resistant to Adversarial Attacks (ICLR 2018 · similarity 0.91)
- Adversarial Risk and the Dangers of Evaluating Against Weak Attacks (ICML 2018 · similarity 0.91)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.