Research map / Adversarial robustness and security / Certified and empirical robustness
Evaluating the Adversarial Robustness of Adaptive Test-time Defenses
Francesco Croce, Sven Gowal, Thomas Brunner, Evan Shelhamer, Matthias Hein, Taylan Cemgil
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply an adaptive white-box transfer attack rather than the original weaker transfer evaluation | SODEF on TRADES · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 60.9% -> 52.2% | Hurt |
| Optimize the full close-loop control defense with APGD and BPDA | Close-loop control · CIFAR-10, ℓ∞ ε=2/255 | robust accuracy: 34.5% -> 5.6% | Hurt |
| Attack the full HD defense adaptively using BPDA and EOT | HD · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 63.0% -> 61.0%; 59.2% -> 59.2%; 45.0% -> 45.2% | Mixed |
| Transfer APGD perturbations from the base classifier to Anti-Adversaries | Anti-Adversaries · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 63.7% | No clear effect |
| Use APGD with BPDA and gradients averaged over purification iterates | SOAP · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 51.02% -> 3.7% | Hurt |
| Allow gradients through the ATLD+ purification step and attack with APGD | ATLD+ · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 65.07% -> 12.6% | Hurt |
| Attack AID-purifier directly with APGD and BPDA instead of transferring attacks | AID-purifier · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 49.3% -> 43.8% | Hurt |
| Attack Natural Supervision purification with BPDA and EOT | Natural Supervision purification · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 59.4% -> 58.4% | Hurt |
| Fix the random seed and remove EOT (ablation) | Natural Supervision purification · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 58.4% -> 56.4% | Hurt |
| Keep negative images separate and hidden from the attacker (ablation) | Natural Supervision purification · CIFAR-10 | robust accuracy | No clear effect |
| Attack ADP directly with APGD and 200 EOT iterations | ADP · CIFAR-10, ℓ∞ ε=8/255 | robust accuracy: 69.71% -> 33.7% ± 0.54 | Hurt |
| Attack AID-purifier on SVHN with a larger APGD budget (ablation) | AID-purifier · SVHN, ℓ∞ ε=12/255 | robust accuracy: 29.1% -> 25.0% | Hurt |
Closest papers by meaning
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks (ICML 2020 · similarity 0.91)
- Benchmarking Adversarial Robustness on Image Classification (CVPR 2020 · similarity 0.90)
- On the Adversarial Vulnerability of Label-Free Test-Time Adaptation (ICLR 2025 · similarity 0.90)
- Practical Evaluation of Adversarial Robustness via Adaptive Auto Attack (CVPR 2022 · similarity 0.90)
- On the Adversarial Risk of Test Time Adaptation: An Investigation into Realistic Test-Time Data Poisoning (ICLR 2025 · similarity 0.88)
- Robust Overfitting Does Matter: Test-Time Adversarial Purification With FGSM (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
