Research map / Adversarial robustness and security / Certified and empirical robustness
Adversaries With Incentives: A Strategic Alternative to Adversarial Robustness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes strategic training, which defends against opponents who maximize their own utility within an incentive uncertainty set, interpolating between clean training and adversarial training.
Problem
Adversarial training defends against maximally harmful opponents for any wrong prediction, which is overly conservative and costs clean accuracy and generalization when real opponents have more specific incentives.
Approach
Model opponents as strategic agents maximizing their own utility over label-dependent targets rather than as zero-sum adversaries, and train against a reduced incentive uncertainty set U; the full set recovers adversarial robustness and the empty set recovers clean training.
Key results
- On CIFAR-10 with ResNet, strategic training improves strategic accuracy over adversarial training by about +8 points against 1-hot opponents (per the appendix table summary).
- Against worst-case anti-semantic opponents on GTSRB with VGG, strategic training reaches 71.2 strategic accuracy vs 63.2 for adversarial training's strategic accuracy; adversarial training scores 43.8 on adversarial attacks.
- Deflection rates of strategic training against successful adversarial attacks can reach about 40%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train against semantic opponents instead of all adversarial targets | ResNet · CIFAR-10 semantic | strategic accuracy: 49.6 -> 52.5 | Helped |
| Train against semantic opponents instead of all adversarial targets | ResNet · CIFAR-10 semantic | clean accuracy: 79.8 -> 78.9 | No clear effect |
| Train against anti-semantic opponents instead of all adversarial targets | ResNet · CIFAR-10 anti-semantic | strategic accuracy: 67.4 -> 73.4 | Helped |
| Train against anti-semantic opponents instead of all adversarial targets | ResNet · CIFAR-10 anti-semantic | clean accuracy: 79.8 -> 85.1 | Helped |
| Train against semantic opponents instead of all adversarial targets | ResNet · GTSRB semantic | strategic accuracy: 55.4 -> 64.9 | Helped |
| Train against semantic opponents instead of all adversarial targets | ResNet · GTSRB semantic | clean accuracy: 90.9 -> 92.0 | Helped |
| Train against anti-semantic opponents instead of all adversarial targets | ResNet · GTSRB anti-semantic | strategic accuracy: 73.0 -> 79.1 | Helped |
| Train against anti-semantic opponents instead of all adversarial targets | ResNet · GTSRB anti-semantic | clean accuracy: 90.9 -> 92.0 | Helped |
| Increase risk-mitigation probability epsilon (ablation) | Strategic training with a known 1-hot utility · CIFAR-10, well-specified opponent | strategic accuracy | Hurt |
| Increase risk-mitigation probability epsilon (ablation) | Strategic training with a misspecified 1-hot utility · CIFAR-10, misspecified opponent | accuracy against worst-case 1-hot utility | Helped |
| Use batch size 64 instead of 32 (ablation) | Strategic training, ResNet · GTSRB semantic | strategic accuracy: 58.3 -> 57.0 | Hurt |
| Use batch size 64 instead of 32 (ablation) | Strategic training, ResNet · GTSRB anti-semantic | strategic accuracy: 79.1 -> 78.4 | Hurt |
Surprising findings
- Strategic training trained against preference-order (non-transitive) utilities still retains about 10 points of protection against adversarial attacks.
- Adversarial training's clean accuracy drops by 13.3 points relative to clean training on CIFAR-10 ResNet.
Limitations
- Misspecified incentive sets can reduce robustness against the true opponent, with losses that are reasonable but not negligible.
- Experiments cover only image classification with small architectures and a few datasets; text and other modalities are left for future work.
Experimental setup
Datasets: CIFAR-10 GTSRB CIFAR-100
Benchmarks: CIFAR-10 GTSRB CIFAR-100
Metrics: clean accuracy strategic accuracy adversarial accuracy deflection rate
Baselines: adversarial training clean training
Builds on: adversarial training PGD strategic classification
Closest papers by meaning
- Efficient Adversarial Attacks on High-dimensional Offline Bandits (ICLR 2026 · similarity 0.88)
- Adversarial Regression with Multiple Learners (ICML 2018 · similarity 0.88)
- Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training (NeurIPS 2021 · similarity 0.88)
- Adversarial Risk in the Generative AI Era Necessitates Dropping the Small Epsilon Ball (NeurIPS 2026 · similarity 0.88)
- Adversarial Policy Learning in Two-player Competitive Games (ICML 2021 · similarity 0.88)
- Adversarial Training and Provable Defenses: Bridging the Gap (ICLR 2020 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.