Research map / Multi-agent systems and games / LLM agents

DRIFT-BENCH: Diagnosing CoopeRative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DRIFT-BENCH, a multi-turn benchmark that injects four classes of input faults (intention, premise, parameter, expression) into state-oriented (AgentBench OS/DB) and service-oriented (StableToolBench G1) tasks, with a persona-driven LLM user simulator and the RISE evaluation protocol.

Problem

LLM agents are typically evaluated on well-specified instructions (the "Oracle Assumption"), so they do not measure multi-turn disambiguation of flawed user inputs under grounded execution risk.

Approach

A theory-grounded (Grice, Austin, Watzlawick) taxonomy of four cooperative-breakdown fault types, injected into verified agent tasks across white-box state environments and black-box API environments, with five GDMS-based user personas and a RISE protocol (Robustness, Intelligence, Safety, Efficiency) that jointly scores task outcome, clarification gain, safe action rate, and interaction rounds.

Key results

• Under input faults without clarification, average performance degradation reaches about 44% (intention), 41% (premise), 50% (parameter) and 46% (expression) on state-oriented tasks.
• Clarification yields gains up to about +23 points on state-oriented tasks, but causes drops on service-oriented tasks (e.g., -25.12% for expression faults per the text).
• Safe action rate is roughly 60% for intention faults but about 29% for premise and parameter faults, meaning agents often execute high-risk actions instead of deferring.

Surprising findings

• Multi-turn clarification degrades performance on service-oriented (black-box API) tasks, termed a 'Clarification Paradox'.
• Agents proceed with high-risk actions in over 70% of premise and parameter fault cases instead of deferring to clarify.

Limitations

• Service-oriented evaluation uses only the G1 single-tool level and state tasks come from OS/DB subsets, limiting scale (200 + 150 tasks).
• User simulator and perturbation generation rely on LLMs (GPT-4o, Gemini-2.0-Flash, Llama-3.3-70B), and persona labels rely on LLM-based assignment.
• Standard deviations are high and the persona correlation analysis uses only n=7 models.

Experimental setup

Datasets: AgentBench (DB, OS) StableToolBench (G1)

Benchmarks: DRIFT-BENCH (AgentBench DB/OS, StableToolBench G1)

Metrics: success score performance degradation (PD) clarification gain (G) safe action rate (SAR) average interaction rounds (AIR)

Baselines: ToolBench AgentBench StableToolBench WebArena GAIA tau-Bench tau^2-Bench ClarifyMT-Bench UserBench IN3

Builds on: AgentBench StableToolBench GDMS

Closest papers by meaning

• COOP$^2$: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems (NeurIPS 2026 · similarity 0.88)
• AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems? (ICLR 2026 · similarity 0.88)
• AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents (NeurIPS 2024 · similarity 0.88)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems (ICLR 2026 · similarity 0.88)
• Talk, Evaluate, Diagnose: User-aware Agent Evaluation with Automated Error Analysis (ICLR 2026 · similarity 0.88)
• Harbor Adapters and Harbor-Mix: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation (NeurIPS 2026 · similarity 0.88)