Research map / Multi-agent systems and games / LLM agents

Think Twice Before You Act: Protecting LLM Agents Against Tool Description Poisoning via Isolated Planning

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that cross-tool tool-description poisoning bypasses existing defenses on AgentDojo and proposes Tool-Guard, which isolates planning over partitioned tool lists and validates tool calls before execution to cut attack success to near zero.

Problem

Cross-tool description poisoning lets an attacker poison non-sensitive tools' descriptions to steer an LLM agent into invoking safety-critical tools, even when the poisoned tool is never selected; existing prompt-injection and tool-filtering defenses transfer poorly to this threat.

Approach

Dynamically partition the tool set into an Influenced list and an Others list, plan separately over each isolated subset so poisoned descriptions cannot steer tools in the other partition, and validate each chosen call (alignment and argument grounding) before execution; failed validation moves the tool into the Influenced list and triggers replanning.

Key results

• On AgentDojo with GPT-4o, Tool-Guard reduces average ASR from 43.30% to 2.06% while attack utility rises from 54.64% to 58.76%.
• Under adaptive PAIR attacks on AgentDojo with GPT-4o-mini, ASR with defense is 2.06%, and 0.00% under semantic-adaptive and TAP attacks.
• Token overhead is about 1.45x benign and 1.39x under attack relative to no defense on AgentDojo with GPT-4o-mini, and latency is about 3.7x no-defense.

Surprising findings

• Baseline ASR decreased when adaptive semantic 'verification' text was appended, apparently because the extra tokens diluted the malicious instruction.
• Across models, more advanced models did not necessarily yield better security (GPT-4o had higher ASR than GPT-4o-mini).

Limitations

• Validator is LLM-based and can misclassify benign tasks semantically close to attack targets, causing utility loss (failure cases in Slack and Workspace).
• Evaluated mainly on AgentDojo and ASB with a self-constructed poisoning setup; the attack scope is limited to appended/authority-style poisoning strategies.
• Latency and token overhead are higher than some baselines such as Progent, and utility under attack still drops on some suites.

Experimental setup

Datasets: AgentDojo ASB

Benchmarks: AgentDojo ASB

Metrics: attack success rate benign utility attack utility token cost latency

Baselines: Tool Filtering Repeat User Prompt System Prompt Drift Progent

Builds on: AgentDojo Progent DRIFT PAIR TAP

Closest papers by meaning

• Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools (NeurIPS 2025 · similarity 0.90)
• MCPTox: A Benchmark for Tool Poisoning on Real-World MCP Servers (AAAI 2026 · similarity 0.88)
• Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents (ICML 2026 · similarity 0.88)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents (NeurIPS 2025 · similarity 0.87)
• A-MemGuard: A Proactive Defense Framework For LLM-Based Agent Memory (ICML 2026 · similarity 0.87)
• SecureClaw: Clawing Back Control of LLM Agents (NeurIPS 2026 · similarity 0.87)