Research map / Adversarial robustness and security / Certified and empirical robustness
A Closer Look at Accuracy vs. Robustness
Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov, Kamalika Chaudhuri
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dropout to TRADES (beta=6) (ablation) | TRADES · SVHN | adv test accuracy: 58.12 -> 63.24 | Improved |
| add dropout to RST (lambda=2) (ablation) | RST · SVHN | test accuracy: 92.39 -> 95.19 | Improved |
| add dropout to AT (ablation) | AT · CIFAR-10 | adv test accuracy: 43.51 -> 43.07 | No clear effect |
| add dropout to RST (lambda=2) (ablation) | RST · CIFAR-10 | adv test accuracy: 41.75 -> 40.24 | Hurt |
| add dropout to TRADES (beta=6) (ablation) | TRADES · CIFAR-10 | adv test accuracy: 48.58 -> 52.32 | Improved |
| add dropout to robust training methods (ablation) | AT/RST/TRADES · CIFAR-10 | test lipschitz: AT 26.23 -> 31.59; TRADES(beta=6) 13.05 -> 8.13 | Mixed |
| add dropout to TRADES (beta=6) (ablation) | TRADES · CIFAR-10 | adv generalization gap: 42.65 -> 26.49 | Improved |
| increase TRADES beta from 1 to 6 (ablation) | TRADES · CIFAR-10 | adv test accuracy: 43.66 -> 48.58 | Improved |
| increase TRADES beta from 1 to 6 (ablation) | TRADES · MNIST CNN1 | test accuracy: 99.26 -> 97.54 | Hurt |
| use AT instead of natural training | WRN-40-10 · CIFAR-10 | test accuracy: 93.81 -> 83.51 | Hurt |
Closest papers by meaning
- Improved techniques for deterministic l2 robustness (NeurIPS 2022 · similarity 0.90)
- A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking (IJCV 2024 · similarity 0.90)
- Unlocking Deterministic Robustness Certification on ImageNet (NeurIPS 2023 · similarity 0.90)
- Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models (ECCV 2018 · similarity 0.89)
- 1-Lipschitz Layers Compared: Memory Speed and Certifiable Robustness (CVPR 2024 · similarity 0.88)
- A Recipe for Improved Certifiable Robustness (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
