Research map / Multi-agent systems and games / LLM agents

AIR: Improving Agent Safety through Incident Response

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces AIR, a DSL-driven incident response framework that detects, contains, recovers from, and eradicates (via synthesized guardrail rules) incidents in LLM agent execution loops, evaluated on code, embodied, and computer-use agents with GPT-5.

Problem

Current LLM agent safety mechanisms focus on preventing failures in advance and offer little support for detecting, containing, recovering from, or preventing recurrence of incidents that occur during execution.

Approach

A DSL with trigger, natural-language check, and remediate components integrated into the agent loop: the agent itself evaluates semantic incident checks against environment state, executes containment/recovery via its tools, and then synthesizes plan-level guardrail rules to block recurrence.

Key results

• AIR detects all executed incidents in most code-agent categories and in SafeAgentBench embodied hazards; remediation success 92.308% for embodied and 97.727% for CUA with AIR vs 80.952% and 76.596% for self-remediation.
• LLM-generated rules achieve 84.560% detection, 91.023% eradication on code agent; 95.973% detection on embodied; 88.636% detection on CUA.
• Guardrail rules reduce incidents across rounds: code agent 54 to 6 to 0, embodied 67 to 5 to 2, CUA 8 to 1 to 0.

Surprising findings

• LLM-generated rules reach detection and eradication rates comparable to developer-authored rules, though some are overly abstract or propose infeasible remediation.
• Adding AIR increases safe-task execution time by about 1.4x for embodied and CUA agents, despite zero false positives.

Limitations

• Relies on the agent's own reasoning to evaluate checks and execute remediation, so reliability degrades on complex tasks.
• Manually authored rules in main evaluation; LLM-generated rules sometimes overly tailored, abstract, or infeasible without human validation.
• Evaluation uses only three agent types and limited task counts; safe-task false positive tests are small (50 and 35 tasks), and CUA execution success is low.

Experimental setup

Datasets: RedCode SafeAgentBench RiOSWorld OSWorld

Benchmarks: RedCode SafeAgentBench RiOSWorld OSWorld

Metrics: detection success rate remediation success rate eradication success rate check time response time false positive count incident count across rounds

Baselines: self-remediation (ablation)

Builds on: CodeAct AgentSpec OpenAI Agent SDK ANTLR4

Closest papers by meaning

• AgentAuditor: Human-level Safety and Security Evaluation for LLM Agents (NeurIPS 2025 · similarity 0.87)
• ORBIT: A Framework for Multi-Agent Security Evaluations (NeurIPS 2026 · similarity 0.87)
• Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (ICLR 2025 · similarity 0.87)
• On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents (ICML 2025 · similarity 0.87)
• LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments (NeurIPS 2026 · similarity 0.87)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems (ICLR 2026 · similarity 0.86)