Research map / Multi-agent systems and games / LLM agents
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Agent Security Bench (ASB), a benchmark formalizing prompt injection, memory poisoning, plan-of-thought backdoor, and mixed attacks plus 11 defenses for LLM agents across 10 scenarios, 13 backbones, and 7 metrics, and proposes the NRP utility-security metric.
Problem
Existing benchmarks evaluate LLM agent security only narrowly (e.g., mostly indirect prompt injection in few scenarios), so attacks and defenses across the agent's operational stages are not comprehensively assessed.
Approach
A unified benchmark and formal threat model covering DPI, IPI, memory poisoning, a training-free Plan-of-Thought backdoor in the system prompt, and mixed attacks, evaluated across many scenarios, backbones, and defenses, plus a Net Resilient Performance metric combining utility and robustness.
Key results
- Mixed Attack achieves the highest average attack success rate of 84.30% across 13 backbones.
- Memory Poisoning is least effective with an average ASR of 7.92%.
- Paraphrasing and Dynamic Prompt Rewriting reduce DPI ASR only to averages of 56.87% and 44.45%, versus 78.38% undefended, so current defenses remain largely ineffective.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine DPI, IPI, and Memory Poisoning rather than use the benchmark's individual attack types | LLM-based agents · ASB | ASR: 84.30% average ASR | Helped |
| Use Memory Poisoning rather than the benchmark's other attack types | LLM-based agents · ASB | ASR: 7.92% average ASR | Hurt |
| Add delimiters around queries | DPI defenses across 13 LLM backbones · ASB, DPI | ASR: 78.38% -> 79.08% | Hurt |
| Paraphrase the query | DPI defenses across 13 LLM backbones · ASB, DPI | ASR: 78.38% -> 56.87% | Helped |
| Add instructional prevention | DPI defenses across 13 LLM backbones · ASB, DPI | ASR: 78.38% -> 76.77% | Helped |
| Rewrite the input query dynamically | DPI defenses across 13 LLM backbones · ASB, DPI | ASR: 78.38% -> 44.45% | Helped |
| Add delimiters around queries | IPI defenses across 13 LLM backbones · ASB, IPI | ASR: 27.98% -> 24.96% | Helped |
| Add instructional prevention | IPI defenses across 13 LLM backbones · ASB, IPI | ASR: 27.98% -> 26.31% | Helped |
| Add sandwich prevention to tool responses | IPI defenses across 13 LLM backbones · ASB, IPI | ASR: 27.98% -> 28.04% | Hurt |
Surprising findings
- Stronger backbones can show lower ASR due to higher refusal rates; GPT-4o has 20.05% refusal under DPI yet 60.35% ASR, while GPT-3.5 Turbo reaches 98.40%.
- Most agents perform below the backbone LLM's leaderboard quality, except Claude-3.5 Sonnet, LLaMA3-70B, and GPT-4o.
Limitations
- Defenses evaluated are mostly prevention-based prompt-level methods and show limited effectiveness.
- Memory poisoning treats the memory database as black box and yields low ASR, so its impact is limited in the reported setting.
Experimental setup
Datasets: ASB scenarios with 400 attack tools and 400 attack tasks
Benchmarks: Agent Security Bench (ASB)
Metrics: ASR RR PNA BP FNR FPR NRP
Baselines: InjecAgent AgentDojo Naive Attack Escape Characters Context Ignoring Fake Completion Delimiters
Builds on: ReAct BadChain InjecAgent AgentDojo chain-of-thought
Closest papers by meaning
- SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks (NeurIPS 2025 · similarity 0.91)
- AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks (ICML 2026 · similarity 0.91)
- Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents (ICLR 2026 · similarity 0.91)
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents (ICLR 2025 · similarity 0.91)
- CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities (ICML 2025 · similarity 0.91)
- Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.