Research map / Adversarial robustness and security / Certified and empirical robustness

Can Go AIs Be Adversarially Robust?

AAAI 2025 · Adversarial robustness and security › Certified and empirical robustness · approach: Attack evaluation and defense critique · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Tests three defenses (positional adversarial training, iterated adversarial training, and a ViT backbone) for KataGo and finds all remain exploitable by freshly trained cyclic adversaries.

Problem

Superhuman Go AIs such as KataGo can be beaten by simple adversarial policies (especially cyclic attacks); the question is whether natural defenses can make them worst-case robust.

Approach

Evaluate three natural defenses against KataGo's cyclic exploit: adversarial training on hand-constructed cyclic positions, iterated adversarial training alternating victim and adversary updates, and replacing the CNN backbone with a vision transformer; then train fresh adversaries to test each defense, and define human-, training-compute-, and inference-compute-robustness criteria.

Key results

• Gift adversary beats dec23-victim 91% at 512 visits; cyclic adv beats may24-victim 56% at 65,536 visits.
• Iterated defense enseiter-9 is beaten 81% by atari-adversary at 512 visits using under 5% of defender compute, and 42% by attack-9 at 65,536 visits.
• ViT-victim at 65,536 visits loses 78% of games to ViT-adversary, showing ViT backbones are also cyclically vulnerable.

Measured effects reported in the paper

Surprising findings

• Adversarial training on cyclic positions did not eliminate the cyclic vulnerability: fine-tuned adversaries still win, even against the defended victims.
• A ViT-based Go AI trained from scratch remains vulnerable to the original cyclic attack, which transfers zero-shot at 2.5% win rate.

Limitations

• Defenses make attacks costlier but none make them impossible.
• Only KataGo-based victims and a single cyclic attack family were tested for generalization.
• Robustness definitions are partly informal and compute bounds are only upper bounds.

Experimental setup

Datasets: KataGo self-play games hand-constructed cyclic positions

Benchmarks: adversary win rate against KataGo victims at varying visit counts

Metrics: win rate compute fraction

Baselines: base-adversary base-victim KataGo dec23-victim KataGo may24-victim ELF OpenGo Leela Zero

Builds on: KataGo adversarial policies (Wang et al. 2023) AlphaZero victim-play

Closest papers by meaning

• Adversarial Policies Beat Superhuman Go AIs (ICML 2023 · similarity 0.93)
• Are AlphaZero-like Agents Robust to Adversarial Perturbations? (NeurIPS 2022 · similarity 0.88)
• Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL (ICLR 2024 · similarity 0.84)
• Rethinking the Adversarial Robustness of Multi-Exit Neural Networks in an Attack-Defense Game (CVPR 2025 · similarity 0.84)
• Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies (ICLR 2024 · similarity 0.83)
• Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees (ICML 2025 · similarity 0.83)