Research map / Multi-agent systems and games / LLM agents

Talk, Evaluate, Diagnose: User-aware Agent Evaluation with Automated Error Analysis

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Software engineering agent benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes the TED framework for user-aware agent evaluation with reusable expert/non-expert personas, grading-note LLM-as-a-judge scoring, turn-aware progress metrics (AUC, PPT, MaxProgressRate@k), and an automated error analysis tool, evaluated on τ²-bench airline/retail and ToolSandbox.

Problem

Agent evaluation lacks a unified, scalable method across heterogeneous domains, and existing approaches ignore the user's role and expertise and stop at coarse metrics without error diagnosis.

Approach

Decouple user persona (expert vs non-expert, generic templates) from task instructions in a simulated user proxy; represent subgoals as natural-language grading notes scored by an LLM judge; add turn-aware metrics (AUC, progress-per-turn) and max-over-trials progress; and cluster judge/agent inconsistencies into high-level error types that inform agent prompt fixes.

Key results

• On tau2-bench airline easy, gpt-4o-mini and mistral-large differ by about 10% in MaxAUC@k (0.85 vs 0.96) despite similar MeanProg, with a ranking change.
• Error Insertion and Human Notes improve agent metrics on ToolSandbox for most setups; gpt-4.1 with Human Notes gains +7-10% MaxPPT@k on ToolSandbox.
• Human-LLM judge agreement (Cohen's Kappa) ranges 0.60-0.92 across settings, highest for gpt-5 agent (0.84 airline, 0.92 ToolSandbox).

Surprising findings

• MaxProgressRate@k saturates and fails to distinguish models on easy samples, while turn-aware AUC reveals differences.
• Rankings between models change under TED versus the original tau2-bench and ToolSandbox evaluation paradigms, e.g., mistral models match or beat gpt-4.1 with expert users.

Limitations

• Judge cannot verify database state changes absent from the trajectory, so silent failures may be missed.
• Only two benchmarks and mostly task-oriented domains; metrics not validated on open-ended dialogue.
• Human validation agreement is lower (kappa 0.60-0.71) for gpt-4.1 agent settings.

Experimental setup

Datasets: tau2-bench airline tau2-bench retail ToolSandbox

Benchmarks: tau2-bench airline tau2-bench retail ToolSandbox

Metrics: MeanProgress@k MaxProgressRate@k MaxAUC@k MaxPPT@k pass@k pass^k Cohen's Kappa

Baselines: tau2-bench original evaluation ToolSandbox milestone score HiTEC-ICL

Builds on: tau-bench ToolSandbox AgentBoard HiTEC-ICL pass@k

Closest papers by meaning

• Agent-as-a-Judge: Evaluate Agents with Agents (ICML 2025 · similarity 0.89)
• Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents (NeurIPS 2026 · similarity 0.89)
• ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (ICLR 2024 · similarity 0.89)
• DRIFT-BENCH: Diagnosing CoopeRative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction (ICML 2026 · similarity 0.88)
• How can we assess human-agent interactions? Case studies in software agent design (ICML 2026 · similarity 0.88)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation (ICLR 2026 · similarity 0.88)