Research map / Adversarial robustness and security / Certified and empirical robustness
Identifying Model Weakness with Adversarial Examiner
Michelle Shu, Chenxi Liu, Weichao Qiu, Alan Yuille
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase RL examination iterations from T=0 to T=500 | AlexNet · ShapeNet object classification | average post-softmax probability on true class: 63.98% -> 2.27% | Hurt |
| increase BO examination iterations from T=0 to T=500 | AlexNet · ShapeNet object classification | average post-softmax probability on true class: 60.05% -> 25.43% | Hurt |
| increase RL examination iterations from T=0 to T=500 | ResNet34 · ShapeNet object classification | average post-softmax probability on true class: 69.03% -> 13.13% | Hurt |
| increase BO examination iterations from T=0 to T=500 | ResNet34 · ShapeNet object classification | average post-softmax probability on true class: 64.19% -> 45.55% | Hurt |
| use BO instead of RL for the first 100 examination iterations | AlexNet · ShapeNet object classification | average post-softmax probability on true class: 65.91% -> 43.58% | Hurt |
| use RL instead of BO after 500 examination iterations | AlexNet · ShapeNet object classification | average post-softmax probability on true class: 25.43% -> 2.27% | Hurt |
| reduce training images per object from m=10 to m=1 (ablation) | RL examiner with ResNet34 · ShapeNet object classification, T=200 | average model performance: 63.81% -> 18.92% | Hurt |
| reduce training images per object from m=10 to m=1 (ablation) | BO examiner with ResNet34 · ShapeNet object classification, T=200 | average model performance: 49.79% -> 10.92% | Hurt |
| reorder the nine sampled rendering factors (ablation) | RL examiner · ShapeNet object classification, lamp examples | convergence to identified weaknesses | No clear effect |
| flip the objective from maximizing loss to minimizing loss (ablation) | Adversarial examiner · ShapeNet piano instance | post-softmax probability on true class | Improved |
Closest papers by meaning
- Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples (NeurIPS 2022 · similarity 0.88)
- Simulated Adversarial Testing of Face Recognition Models (CVPR 2022 · similarity 0.87)
- Increasing Confidence in Adversarial Robustness Evaluations (NeurIPS 2022 · similarity 0.86)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.85)
- On the Adversarial Risk of Test Time Adaptation: An Investigation into Realistic Test-Time Data Poisoning (ICLR 2025 · similarity 0.84)
- Doppelgangers and Adversarial Vulnerability (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
