Research map / Multi-agent systems and games / LLM agents

ToolACE-MT: Non-Autoregressive Generation for Agentic Multi-Turn Interaction

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ToolACE-MT, a non-autoregressive iterative framework (skeleton initialization, mask-and-fill refinement, offline verification) that builds multi-turn function-calling dialogues more cheaply than autoregressive multi-agent simulation.

Problem

Generating high-quality multi-turn, multi-step agentic tool-use dialogues via autoregressive multi-agent simulation is computationally costly, hard to control in complexity, and lacks global consistency.

Approach

Generate whole conversational trajectories non-autoregressively: first a structurally complete coarse skeleton, then iterative turn-level mask-and-fill refinement (complexity injection and LLM-judged reasonability refinement), then rule- and model-based offline verification.

Key results

• On BFCL-v3, the NAIG-style method reaches 40.25% multi-turn accuracy vs 31.38% for multi-agent simulation and 9.25% for the Llama3.1-8B-Inst base.
• On tau-Bench average Pass@1 rises to 20.6% vs 15.9% for multi-agent simulation; ACEBench Agent PA improves to 34.0% vs 15.0%.
• Generation requires 188k API calls vs 275k for MAS to obtain 8,000 valid samples (offline verification pass rate 72.3% vs 61.1%).

Surprising findings

• Base Llama3.1-8B-Inst scores higher (26%) than all trained models on the tau-Bench Airline domain, attributed to evaluation artifacts with empty-action ground truths.
• Gains from training on the generated data are smaller for Qwen2.5-7B and Qwen3-8B than for Llama3.1-8B, likely because these newer models already include agentic training.

Limitations

• Generation with the smaller GPT-4o-mini yields much lower verification pass rate (48.7%) and more API calls, and Llama3.1-8B failed to produce valid instances, so the pipeline depends on a strong generator LLM.
• Parameter-related errors remain the dominant failure mode in offline verification.
• Live single-turn gains are smaller than MAS, as trained models ask more clarification questions on ambiguous queries.

Experimental setup

Datasets: ToolACE tool pool (actual and simulated tools) ToolACE-10k Toucan-1.5M ToolDial-9k

Benchmarks: BFCL-v3 tau-Bench ACEBench

Metrics: accuracy pass@1 end-to-end accuracy process accuracy semantic similarity entailment rate entropy Distinct-3 pass rate API calls

Baselines: Multi-Agent Simulation GPT-4o-2024-11-20 Gemini-2.5-Pro DeepSeek-V3-0324 Llama3.1-70B-Inst Llama3.1-8B-Inst ToolACE-10k Toucan-1.5M ToolDial-9k

Builds on: Mask-Predict multi-agent simulation data synthesis (ToolACE) LoRA

Closest papers by meaning

• T1: A Tool-Oriented Conversational Dataset for Multi-Turn Agentic Planning (NeurIPS 2025 · similarity 0.91)
• ToolDial: Multi-turn Dialogue Generation Method for Tool-Augmented Language Models (ICLR 2025 · similarity 0.88)
• APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay (NeurIPS 2025 · similarity 0.88)
• AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.88)
• AutoTool: Efficient Tool Selection for Large Language Model Agents (AAAI 2026 · similarity 0.87)
• Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models (NeurIPS 2024 · similarity 0.87)