Research map / Multi-agent systems and games / LLM agents

ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning

ICML 2025 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ShieldAgent, an LLM-based guardrail agent that verifies web agent action trajectories against policy-derived probabilistic logic rule circuits, and releases ShieldAgent-Bench (3K safety-labeled trajectories across 6 web environments and 7 risk categories).

Problem

Autonomous LLM web agents are vulnerable to malicious instructions and attacks, and existing guardrails designed for LLM outputs do not account for sequential action trajectories or explicit, lengthy safety policies.

Approach

Extract verifiable LTL-style rules from policy documents, organize them into action-based probabilistic rule circuits (Markov Logic Networks), and decide safety via a relative probability condition on the invoked action, with a tool-based shielding plan that assigns predicate truth values and runs formal verification.

Key results

• On ShieldAgent-Bench, ShieldAgent reaches 90.4% ACC (agent-based) and 91.7% ACC (environment-based) with FPR of 5.6% and 4.0%, and ARR of 87.5% and 92.7%.
• ShieldAgent reduces GPT-4o queries by 64.7% and inference time by 58.2% relative to baselines, per the paper's claims.
• On existing benchmarks, ShieldAgent achieves 91.1% ACC on ST-WebAgentBench, 94.1% on VWA-Adv, and 86.9% on AgentHarm.

Measured effects reported in the paper

Surprising findings

• Rule Traverse with GPT-4o is more costly than direct prompting but its accuracy gain is smaller than ShieldAgent's, and direct prompting has the lowest FPR on some benchmarks.

Limitations

• Relies on GPT-4o for policy extraction, rule extraction and predicate refinement, so quality depends on a closed-source model.
• Benchmark attacks are adapted to a single web agent (AWM), limiting generality.
• Human expert review is needed to finalize rules, and the bench labels are human-reviewed only as a quality control step.

Experimental setup

Datasets: ST-WebAgentBench VWA-Adv AgentHarm WebArena instruction templates

Benchmarks: ShieldAgent-Bench ST-WebAgentBench VWA-Adv AgentHarm

Metrics: accuracy false positive rate average rule recall rate number of API queries inference time

Baselines: Direct prompting (GPT-4o) Rule Traverse GuardAgent LlamaGuard VWA-Adv guardrail baselines

Builds on: GuardAgent AWM AgentPoison AdvWeb Markov Logic Network

Closest papers by meaning

• GuardAgent: Safeguard LLM Agents via Knowledge-Enabled Reasoning (ICML 2025 · similarity 0.91)
• AgentAuditor: Human-level Safety and Security Evaluation for LLM Agents (NeurIPS 2025 · similarity 0.87)
• LiSA: Lifelong Safety Adaptation via Conservative Policy Induction (NeurIPS 2026 · similarity 0.87)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data (ICLR 2026 · similarity 0.87)
• SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety (ICML 2026 · similarity 0.86)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents (NeurIPS 2025 · similarity 0.86)