Research map / Multi-agent systems and games / LLM agents

TUMIX: Multi-Agent Test-Time Scaling with Tool-Use Mixture

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TUMIX, a multi-agent test-time scaling framework that runs 15 tool-using agents (text, code interpreter, search) in parallel over multiple refinement rounds with LLM-judged early termination, improving Gemini-2.5 accuracy on HLE, GPQA and AIME.

Problem

Tool-augmented LLM reasoning lacks guidance on how to combine textual reasoning, code execution, and web search for diverse questions, and test-time scaling methods don't exploit these tools well.

Approach

Run a diverse pool of text-only and tool-augmented agents in parallel, iteratively share and refine answers across rounds, use LLM-as-judge to stop early at about 49% of inference cost, and optionally have LLMs auto-design additional agents.

Key results

• TUMIX reaches 32.3% HLE, 87.9% GPQA and 96.7% AIME with Gemini-2.5-Pro, average normalized score 72.3 vs 69.4 for best baseline (GSA/SciMaster/Self-MoA range).
• TUMIX with Gemini-2.5-Flash reaches average 60.6 vs 55.5 for best baseline DEI.
• LLM-based termination retains peak accuracy at about 49% of inference cost; TUMIX+ raises Gemini-2.5-Pro HLE to 34.1%.

Surprising findings

• Coverage decreases monotonically across refinement rounds, meaning correct answers are often discarded during refinement, and GPQA accuracy declines after round 2.
• Diverse heterogeneous agent groups outperform repeatedly sampling the single best agent, contrary to Self-MoA's conclusion.

Limitations

• Test-time scaling demands many inferences and roughly two orders of magnitude more tokens.
• LLMs tend to be overconfident and terminate early, requiring a minimum of two rounds.
• SciMaster's tools are not open-sourced, so its reported numbers could not be fully reproduced.

Experimental setup

Datasets: HLE GPQA Diamond AIME 2024 AIME 2025

Benchmarks: HLE GPQA Diamond AIME 24&25

Metrics: success rate coverage average normalized accuracy inference cost token count

Baselines: Majority-Vote GSA Self-Reflection SETS Self-MoA Symbolic-MoE DEI SciMaster

Builds on: Mixture-of-Agents Self-MoA SciMaster Symbolic-MoE DEI

Closest papers by meaning

• AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Tasks (NeurIPS 2025 · similarity 0.89)
• A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.89)
• AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering (NeurIPS 2026 · similarity 0.88)
• Fleet of Agents: Coordinated Problem Solving with Large Language Models (ICML 2025 · similarity 0.88)
• ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents (ICLR 2025 · similarity 0.88)
• MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles (ICML 2025 · similarity 0.88)