Research map / Multi-agent systems and games / LLM agents
A2ASecBench: A Protocol-Aware Security Benchmark for Agent-to-Agent Multi-Agent Systems
Tianhao Li, Chuangxin Chu, Yujia Zheng, Bohan Zhang, Neil Zhenqiang Gong, Chaowei Xiao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents A2ASecBench, a protocol-aware security benchmark and first comprehensive security evaluation of agent-to-agent (A2A) multi-agent systems built on LLMs.
Problem
Agent-to-agent protocols enable interoperability among LLM-based multi-agent systems but introduce new security vulnerabilities that are not systematically evaluated.
Approach
A taxonomy of A2A-specific threats paired with a benchmark that evaluates security of A2A-MAS in a protocol-aware way; the abstract is truncated so specifics beyond the taxonomy and benchmark are not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase number of injected lookalike AgentCards from 3 to 7 (ablation) | AgentCard Spoofing · A2A discovery | ASR: 76% -> 87% -> 95% | Improved |
| embed a cloaked agent in the multi-agent system | A2A-MAS benign tasks · Healthcare utility | utility score: 0.872 -> 0.595 | Hurt |
| embed a cloaked agent in the multi-agent system | A2A-MAS benign tasks · Travel utility | utility score: 0.853 -> 0.682 | Hurt |
| embed a cloaked agent in the multi-agent system | A2A-MAS benign tasks · Finance utility | utility score: 0.962 -> 0.749 | Hurt |
| add NeMo Guardrails as a security gateway | A2A-MAS · Agent-Side Request Forgery | ASR: ASR 0.23-0.48 remains | Improved |
| add NeMo Guardrails as a security gateway | A2A-MAS · Half-Open Task Flooding and Artifact-Triggered Script Injection | ASR: HOTF 0.85-0.91, ATSI 0.91-0.94 | No clear effect |
| add NeMo Guardrails as a security gateway | A2A-MAS · Cycle Overflow | ASR: 1.00 -> 0.66-0.73 | Mixed |
| transfer attack patterns to ANP protocol | attack patterns · ANP | ASR: AS 0.98, others 1.00 | Hurt |
Experimental setup
Benchmarks: A2ASecBench
Closest papers by meaning
- Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents (ICLR 2026 · similarity 0.91)
- ORBIT: A Framework for Multi-Agent Security Evaluations (NeurIPS 2026 · similarity 0.90)
- Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (ICLR 2025 · similarity 0.90)
- Shadows in the Code: Exploring the Risks and Defenses of LLM-based Multi-Agent Software Development Systems (AAAI 2026 · similarity 0.89)
- Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agents (ICML 2026 · similarity 0.88)
- AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
