Research map / Multi-agent systems and games / LLM agents

Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces threat snapshots and the b3 benchmark, built from 194,331 crowdsourced attacks, to evaluate the security of 34 backbone LLMs in AI agents.

Problem

Lack of a systematic way to measure how the choice of backbone LLM affects security of AI agents, since existing agent security benchmarks either cover only specific vulnerabilities or require modeling full agents.

Approach

Model each LLM call as a stateless 'threat snapshot' capturing the agent state, attack insertion and scoring for a single LLM vulnerability, then benchmark backbones on crowdsourced adversarial attacks gathered through a gamified red-teaming challenge (Gandalf Agent Breaker).

Key results

• Across 34 LLMs, claude-haiku-4-5 and claude-sonnet-4-5 (reasoning enabled) are the most secure on the total vulnerability score.
• Enabling reasoning generally lowers vulnerability scores; only tiny variants (gemini-2.5-flash-lite, gpt-5-nano) get worse.
• The best open-weights model, kimi-k2-thinking (score 0.34), beats gpt-5.1 without reasoning; model size shows no meaningful effect on security.

Surprising findings

• Model size does not correlate with security; larger non-reasoning variants sometimes perform worse than smaller ones.
• The two highest-capability models (gpt-5.1 and kimi-k2-thinking) rank only 8th and 14th in security.

Limitations

• Only utility and latency were not evaluated; security-utility tradeoff not covered.
• Only single-step threat snapshots; multi-step and multi-agent attacks not benchmarked.
• Strongest 210 attacks withheld from public release, and automated attack generation is not yet strong enough.

Experimental setup

Datasets: crowdsourced adversarial attacks from Gandalf Agent Breaker challenge

Benchmarks: b3 benchmark

Metrics: vulnerability score bootstrap 95% confidence interval

Baselines: claude-haiku-4-5 claude-sonnet-4-5 gpt-5.1 kimi-k2-thinking gpt-5-nano gemini-2.5-flash-lite

Builds on: ReAct Crescendo Inspect framework Gandalf Agent Breaker

Closest papers by meaning

• Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (ICLR 2025 · similarity 0.91)
• AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (NeurIPS 2024 · similarity 0.90)
• AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents (ICLR 2025 · similarity 0.90)
• AgentAuditor: Human-level Safety and Security Evaluation for LLM Agents (NeurIPS 2025 · similarity 0.89)
• OpenAgentSafety: A Comprehensive Framework For Evaluating Real-World AI Agent Safety (ICLR 2026 · similarity 0.89)
• ORBIT: A Framework for Multi-Agent Security Evaluations (NeurIPS 2026 · similarity 0.89)