Research map / Multi-agent systems and games / LLM agents

Optimizing Agent Planning for Security and Autonomy

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Software engineering agent benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes autonomy metrics (HITL load, TCR@k) for deterministic IFC-based agent defenses and Prudentia, an IFC-aware planner that reduces human-in-the-loop approvals while blocking prompt injections on AgentDojo and WASP.

Problem

Deterministic information-flow control defenses against indirect prompt injection reduce task completion, but existing evaluations ignore their benefit of reducing reliance on human-in-the-loop approval for consequential actions.

Approach

Introduces HITL-load and TCR@k autonomy metrics, and an IFC-aware planner that treats policy compliance as an explicit objective via a plan tool, strategic variable expansion, and user endorsement of untrusted data instead of per-call approval.

Key results

• On AgentDojo with o3-mini, Prudentia reaches 59.1% TCR@0 versus 50.1% for FIDES and 24.3% for Basic, with HITL load 16.4 vs 18.8 for FIDES.
• On WASP, Prudentia blocks all prompt injection attacks (0 ASR) and reduces HITL load to 0 across GPT-4o, o1, o3-mini and o4-mini.
• With o4-mini, Prudentia reaches 73.2% TCR at 19.2 HITL load versus FIDES' 75.7% at 36.8, a 1.9x reduction in HITL load.

Surprising findings

• A basic IFC mechanism without autonomy optimization reduces HITL load by 1.5x with no loss in task completion relative to the Basic agent.
• Prudentia's token and cost overhead is substantially higher than Basic-IFC and FIDES, e.g., roughly 3-4x more input tokens than FIDES on AgentDojo.

Limitations

• Assumes the user, planner and tools are trusted and that a human approving consequential calls is secure; HITL interface manipulation is left to future work.
• Overhead in tokens, cost and turns is substantially higher than baselines and not optimized.
• Evaluated only on OpenAI models and two benchmarks; declassification intentionally omitted.

Experimental setup

Datasets: AgentDojo WASP

Benchmarks: AgentDojo WASP

Metrics: task completion rate HITL load TCR@k attack success rate turns token cost

Baselines: Basic Basic-IFC FIDES

Builds on: FIDES CaMeL ReAct Dual LLM pattern

Closest papers by meaning

• AI Agents May Always Fall for Prompt Injections (NeurIPS 2026 · similarity 0.89)
• Architecture Matters for Multi-Agent Security (ICML 2026 · similarity 0.88)
• Securing AI Agents with Information-Flow Control (NeurIPS 2026 · similarity 0.88)
• Attack Selection In Agentic AI Control Evaluations Meaningfully Decreases Safety (NeurIPS 2026 · similarity 0.88)
• CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution (ICML 2026 · similarity 0.87)
• AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (NeurIPS 2024 · similarity 0.87)