Research map / Multi-agent systems and games / LLM agents

GuardAgent: Safeguard LLM Agents via Knowledge-Enabled Reasoning

ICML 2025 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GuardAgent, an LLM agent that safeguards other LLM agents by planning from safety guard requests and generating/executing guardrail code with retrieved demonstrations, plus two new benchmarks (EICU-AC, Mind2Web-SC).

Problem

Existing LLM guardrails only moderate textual harm and cannot enforce diverse, agent-specific safety or access-control requests on LLM agents' actions.

Approach

A guardrail agent does knowledge-enabled task planning from safety guard requests and memory demonstrations, then translates the plan into executable guardrail code calling a extendable toolbox of functions, so enforcement is deterministic and non-invasive to the target agent.

Key results

• GuardAgent with Llama3.3-70B and GPT-4 reaches 98.4-99.1% (Llama3.3 99.1, GPT-4 98.7) LPA on EICU-AC and up to 93.0% (Llama3.3-70B) / 90.0% (GPT-4) LPA on Mind2Web-SC, with 100% FRA in all settings.
• Hardcoded Safety Rules degrade task performance, with FRA as low as 3.2% for EHRAgent with GPT-4.
• LlamaGuard3-8B's label prediction accuracy is about 50% on both benchmarks, close to random guessing.

Measured effects reported in the paper

Surprising findings

• Without toolbox and memory, GPT-4 chose code-based guardrails for 68% of examples even when instructed to produce guardrails in natural language.
• Retrieving least-similar demonstrations instead of most-similar reduces LPA, showing memory relevance matters.

Limitations

• Mind2Web-SC Rule 5 (movies/music/video) accuracy drops with GPT-4 due to broad rule coverage and long queries.
• Evaluated on only two target agents and two benchmarks with fairly small datasets (316 and 200 examples).
• Core LLM capability affects guardrail performance; weaker LLMs not studied.

Experimental setup

Datasets: EICU Mind2Web

Benchmarks: EICU-AC Mind2Web-SC CSQA

Metrics: label prediction accuracy label prediction precision label prediction recall explanation accuracy final response accuracy

Baselines: Model-Guarding-Agent LlamaGuard Hardcoded Safety Rules

Builds on: EHRAgent SeeAct Mind2Web LlamaGuard chain-of-thought

Closest papers by meaning

• ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning (ICML 2025 · similarity 0.91)
• AgentAuditor: Human-level Safety and Security Evaluation for LLM Agents (NeurIPS 2025 · similarity 0.90)
• Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (ICLR 2025 · similarity 0.89)
• SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks (NeurIPS 2025 · similarity 0.88)
• PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety (NeurIPS 2026 · similarity 0.87)
• SecureClaw: Clawing Back Control of LLM Agents (NeurIPS 2026 · similarity 0.87)