Research map / Adversarial robustness and security / Attacks on text and graphs

AutoAdvExBench: Benchmarking Autonomous Exploitation of Adversarial Example Defenses

ICML 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: Adversarial prompts for language models · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces AutoAdvExBench, a benchmark of 75 adversarial-example defenses (24 CTF-like, 51 real-world) to test whether LLM agents can autonomously write attacks, and a four-step agent that breaks 75% of CTF-like but only 13-21% of real-world defenses.

Problem

Whether LLMs can autonomously exploit real-world adversarial example defenses end-to-end, given a defense's code and paper, and how this compares to performance on CTF-style homework implementations.

Approach

A proxy-free, mechanically verifiable benchmark where the task is the actual research task (breaking defenses by producing adversarial examples), paired with a special-purpose four-step agent (forward pass, differentiable pass, FGSM, PGD) that substantially outperforms general-purpose coding agent frameworks.

Key results

• The four-step agent with Claude 3.5 Sonnet breaks 18 of 24 CTF-like defenses (75%) but only 6 of 51 real-world defenses (about 13%).
• Claude 3.7 Sonnet is the best on real-world defenses, attacking 11 of 51 (21%), but breaks only 13 of 24 CTF-like defenses.
• Direct LLM prompting (zero-shot, 8 attempts) breaks zero defenses for Claude 3.5 Sonnet, GPT-4o and o1; with debugging loops, 1 to 3 defenses are broken.

Measured effects reported in the paper

Surprising findings

• Reasoning models (o1, o3-mini, Claude 3.7 Sonnet with thinking) performed worse than non-reasoning counterparts in the agent framework.
• Adding o3-mini as advisor to Sonnet 3.5 every 5 actions reduced attack success rate.
• No model successfully attacked any of the 16 defenses implemented in TensorFlow v1, due to calls to nonexistent library functions.

Limitations

• Possible benchmark contamination since several defenses have published breaks.
• Gradient-free attacks can break many defenses, so white-box attack capability is not always necessary.
• Research code is not representative of production code; results may not generalize to other security domains.

Experimental setup

Datasets: arXiv papers (crawled, filtered) Google self-study course adversarial robustness defenses (24 CTF-like) CIFAR-10 ImageNet MNIST

Benchmarks: AutoAdvExBench

Metrics: attack success rate robust accuracy number of defenses attacked (robust accuracy below half of c

Baselines: Claude 3.5 Sonnet agent GPT-4o agent o1 agent o3-mini agent Claude 3.7 Sonnet thinking SWE-Bench agent scaffolds direct LLM prompting

Builds on: Cybench PGD FGSM Google self-study course in evaluating adversarial robustness

Closest papers by meaning

• Adversarial Reasoning at Jailbreaking Time (ICML 2025 · similarity 0.88)
• An LLM can Fool Itself: A Prompt-Based Adversarial Attack (ICLR 2024 · similarity 0.87)
• REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective (ICML 2025 · similarity 0.87)
• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.87)
• Efficient Adversarial Training in LLMs with Continuous Attacks (NeurIPS 2024 · similarity 0.87)
• Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling (NeurIPS 2026 · similarity 0.87)