Research map / Multi-agent systems and games / LLM agents

ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ASPI, a 728-scenario benchmark built on AgentDojo showing that agents become substantially more vulnerable to prompt injection when they seek clarification, with the ask_user channel driving most of the gap.

Problem

Clarification-seeking is treated as desirable agent behavior, but its security implications are unknown: does entering a clarification state, which opens a new user-associated input channel, increase prompt-injection susceptibility?

Approach

Create matched execution and clarification versions of AgentDojo tasks by deleting one user-provided slot, then deliver the same attacker goal through tool outputs, the ask_user return, or the user's clarification reply, and decompose the ASR gap into state and channel effects.

Key results

• o3 attack success rises from 1.8% under execution tool attacks to 34.0% under clarification ask_user attacks; Gemini-3-Flash rises from 2.2% to 35.7%.
• Kimi K2.5 rises from 11.1% to 63.1% ASR under clarification ask_user attacks versus execution tool attacks.
• Prompt guard defense lowers Gemini-3-Flash clarification ask_user ASR from 35.7% to 27.0%, but a residual gap remains.

Surprising findings

• Interaction state alone has heterogeneous effects: some models show near-zero or negative state effects under tool-channel attacks, while the ask_user channel drives most of the amplification.
• Claude-Opus-4.7 stays near-zero ASR in both settings, and its user-channel ASR drops under clarification.

Limitations

• Ambiguity is synthetic, built from single-slot removal with one clarification round, so real-world ambiguity may be richer.
• Assumes a single external adversary without access to the system prompt and no adaptive attacks.
• Evaluation covers ten models and four AgentDojo domains only.

Experimental setup

Datasets: AgentDojo

Benchmarks: ASPI AgentDojo

Metrics: attack success rate task utility clarification rate paired delta ASR

Baselines: exec_tool exec_user exec_next_turn clarif_tool clarif_user

Builds on: AgentDojo InjecAgent

Closest papers by meaning

• Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks (NeurIPS 2026 · similarity 0.89)
• Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (ICLR 2025 · similarity 0.88)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents (NeurIPS 2025 · similarity 0.87)
• Uncertainty-Aware Clarification in LLM Agents with Information Gain (ICML 2026 · similarity 0.87)
• LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments (NeurIPS 2026 · similarity 0.87)
• AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (NeurIPS 2024 · similarity 0.87)