Research map / Multi-agent systems and games / LLM agents

ORBIT: A Framework for Multi-Agent Security Evaluations

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ORBIT, a configurable Inspect-based framework and benchmark suite for multi-agent safety/security evaluation, and uses it to compare four defenses across misuse, compromised-agent, and collusion threats.

Problem

Multi-agent LLM systems face security threats (prompt injection, compromised agents, collusion) but defense evaluations are bespoke, single-threat, and not comparable across topologies, memory settings, and attacks.

Approach

A six-layer configurable evaluation framework (setup, scenario, attack, defense, execution, evaluation) built on Inspect AI that factorizes topology, memory visibility, attacks, and defenses so one axis can be varied while others are held fixed, plus a multi-agent benchmark suite adapted from single-agent benchmarks.

Key results

• LLM monitor and guardian cut compromised-agent ASR on SWE-Bench multi-issue from 72% to 12%, but collusion ASR stays at 30% under the monitor.
• Dual-LLM incurs a -9.2 point multi-issue resolution cost ([-14.0, -4.5]).
• Undefended standalone agents on BrowserART range from 0% ASR (Sonnet 4.6) to 27% (Llama 70B); Qwen3-235B rises from 17% to 36% under Star orchestration.

Surprising findings

• Per-action defenses that cut compromised-agent ASR by 60 points give no measurable protection against colluding agents.
• No undefended JiraTicket coalition showed measurable advantage over its fair share on any model or topology tested.

Limitations

• Attacks use fixed payload templates rather than adaptive red-teaming, likely overstating static defense effectiveness.
• Evaluates only a representative slice of a combinatorially large configuration space.
• Assumes turn-based discrete message passing; excludes continuous or learned communication protocols.

Experimental setup

Datasets: BrowserART SWE-Bench Verified RedCode-Gen BigCodeBench OSWorld OS-Harm tau2-Bench airline Colosseum MeetSched

Benchmarks: BrowserART RedCode-Gen SWE-Bench Verified (multi-issue) JiraTicket

Metrics: attack success rate benign task completion rate propagation depth regret coalition advantage

Baselines: Terrarium MASEval LangGraph MASTER MASpi ASB MAESTRO Concordia

Builds on: Inspect AI BrowserART SWE-Bench OSWorld tau2-bench

Closest papers by meaning

• Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (ICLR 2025 · similarity 0.90)
• Shadows in the Code: Exploring the Risks and Defenses of LLM-based Multi-Agent Software Development Systems (AAAI 2026 · similarity 0.90)
• Architecture Matters for Multi-Agent Security (ICML 2026 · similarity 0.89)
• Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents (ICLR 2026 · similarity 0.89)
• AgentAuditor: Human-level Safety and Security Evaluation for LLM Agents (NeurIPS 2025 · similarity 0.88)
• AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks (ICML 2026 · similarity 0.88)