Research map / Multi-agent systems and games / LLM agents

Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SupervisorAgent, an LLM-free adaptive filter triggering a supervisor LLM to correct errors, guide inefficient loops, and purify long observations in multi-agent systems, cutting token use ~30% on GAIA with comparable accuracy.

Problem

Multi-agent systems are token-expensive and fragile: errors propagate, agents loop or take verbose paths, and long observations inflate context. Existing work focuses on post-hoc failure attribution rather than runtime intervention.

Approach

A lightweight, non-intrusive meta-agent supervisor intercepts each agent step via an LLM-free heuristic filter (errors, repetitive/inefficient patterns, excessive observation length) and only then invokes an LLM to apply one of four interventions: approve, provide_guidance, correct_observation, or run_verification.

Key results

• On GAIA validation with Smolagent (GPT-4.1), SupervisorAgent reduces average tokens by 29.68% at pass@1 with unchanged accuracy (50.91%).
• On HumanEval, token use drops 23.74% (Qwen3-32B) with slightly higher accuracy (92.68 vs 92.07).
• On AIME, accuracy rises from 30.00 to 36.67 while tokens drop 18.92%.

Surprising findings

• Aggressive observation purification (removing HTML structure and truncation cues) can harm ReAct-style agent performance, suggesting 'noisy' information is a useful signal.
• Removing the purification component yields nearly identical latency to baseline, so most latency overhead comes from purification calls.

Limitations

• Adaptive filter is heuristic and rule-based; thresholds are hand-tuned per benchmark.
• Latency overhead of supervisory calls, roughly 37% increase on GAIA.
• Aggressive purification can overcompress context and cause minor accuracy or F1 drops.

Experimental setup

Datasets: GAIA GSM-hard AIME 2024 HumanEval MBPP DROP

Benchmarks: GAIA GSM-hard AIME 2024 HumanEval MBPP DROP

Metrics: pass@k accuracy F1 average token consumption latency

Baselines: Vanilla LLM CoT Self-Consistency CodeAgent Smolagent OAgents MetaAgent OWL AWorld

Builds on: ReAct Smolagent AWorld OAgents Model Context Protocol

Closest papers by meaning

• AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems? (ICLR 2026 · similarity 0.87)
• Agent Primitives: Reuseable Latent Building Blocks for Multi-Agent Systems (ICML 2026 · similarity 0.87)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems (ICLR 2026 · similarity 0.86)
• Reward Modeling for Multi-Agent Orchestration (NeurIPS 2026 · similarity 0.86)
• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory (ICLR 2026 · similarity 0.86)
• Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs (NeurIPS 2026 · similarity 0.86)