Research map / Large language models / Agents, code and math
AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoRedTeamer, a lifelong multi-agent framework that automatically generates test cases from risk categories or behaviors, combines memory-guided attack selection, and autonomously discovers and implements new jailbreak attacks from recent literature.
Problem
Existing red teaming relies heavily on human-curated test cases and individual attack vectors, lacks coverage of emerging attacks, and needs manual effort to implement new strategies and choose attack configurations.
Approach
A dual-agent system: a red teaming agent that generates seed prompts from high-level risk categories and selects attack combinations via an attack memory tracking per-attack and combination success, plus a strategy proposer agent that retrieves papers via Semantic Scholar, proposes, implements and validates new attacks into a growing library.
Key results
- On HarmBench against Llama-3.1-70B, AutoRedTeamer reaches 0.82 ASR versus 0.67 for AutoDAN-Turbo and 0.60 for PAIR and TAP, with 14 evaluation queries.
- On Claude-3.5-Sonnet, AutoRedTeamer reaches 0.28 ASR while the best baseline (Pliny) reaches 0.14 and most dynamic baselines are near zero.
- On AIR risk categories, AutoRedTeamer outperforms AIR-Bench human-curated test cases across 43 level-3 categories, e.g. Llama-3-Instruct-8B ASR rises from 0.21 to 0.90.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add agent-proposed attacks to the human-initialized attack library (ablation) | AutoRedTeamer · HarmBench, Llama-3.1-70B | ASR: 0.75 -> 0.82 | Helped |
| Use only agent-proposed attacks instead of the full library (ablation) | AutoRedTeamer · HarmBench, Llama-3.1-70B | ASR: 0.82 -> 0.78 | Hurt |
| Use only human-initialized attacks instead of the full library (ablation) | AutoRedTeamer · HarmBench, Llama-3.1-70B | ASR: 0.82 -> 0.75 | Hurt |
| Add memory-based attack selection (ablation) | AutoRedTeamer · HarmBench, GPT-4o | ASR: 0.43 -> 0.69 | Helped |
| Replace random attack selection with memory-guided selection (ablation) | AutoRedTeamer · HarmBench, GPT-4o | ASR: 0.12 -> 0.69 | Helped |
| Generate test cases from risk categories instead of using AIR-Bench human-curated prompts | AutoRedTeamer · AIR categories, Llama-3-Instruct-8B | ASR: 0.21 -> 0.90 | Helped |
Surprising findings
- Static templates such as Pliny vary widely in effectiveness across models (0.91 on Mixtral-8x7B versus 0.14 on Claude-3.5-Sonnet).
- Random attack selection drops ASR to 0.12, far below memory-guided selection at 0.69 on GPT-4o.
Limitations
- Reliance on LLM-based attack implementation and proposals may introduce biases and limit quality.
- Risk of overfitting to model-specific vulnerabilities that may not transfer to other models or real-world scenarios.
- Runtime is longer than PAIR (4h25m vs 1h36m on Llama-3.1-70B).
Experimental setup
Datasets: HarmBench AIR-Bench
Benchmarks: HarmBench AIR-Bench
Metrics: attack success rate queries tokens used diversity (embedding coverage) runtime
Baselines: PAIR TAP Rainbow Teaming AutoDAN-Turbo ArtPrompt Pliny FewShot AIR-Bench
Builds on: PAIR AutoDAN-Turbo Rainbow Teaming HarmBench AIR-Bench
Closest papers by meaning
- Automated Red Teaming with GOAT: the Generative Offensive Agent Tester (ICML 2025 · similarity 0.89)
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (ICML 2024 · similarity 0.89)
- TreeTeaming: Autonomous Red-Teaming of Vision-Language Models via Hierarchical Strategy Exploration (CVPR 2026 · similarity 0.89)
- Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agents (ICML 2026 · similarity 0.88)
- Learning Diverse Attacks on Large Language Models for Robust Red-Teaming and Safety Tuning (ICLR 2025 · similarity 0.88)
- CoP: Agentic Red-teaming for Large Language Models using Composition of Principles (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.