Research map / Multi-agent systems and games / LLM agents

Iterative Critique-and-Routing Controller for Multi-Agent Systems with Heterogeneous LLMs

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an RL-trained controller that critiques intermediate drafts, decides to stop or continue, and re-routes across a heterogeneous LLM agent pool, optimized with GRPO under a Lagrangian usage-constrained reward.

Problem

One-shot routing in multi-agent LLM systems gives no feedback loop to critique and refine drafts, and optimizing answer quality alone over-uses the strongest, most expensive agent.

Approach

Casts controller coordination as a finite-horizon MDP where each action both verifies the previous draft and routes to the next agent (or stops), with a composite routing+verification reward and Lagrangian-relaxed per-agent usage penalties optimized by GRPO.

Key results

• System 1 (Qwen2.5-7B-Base controller): average accuracy 80.5% vs. Router-R1 69.1% across seven math benchmarks.
• System 2 (Qwen3-4B-Base controller): average accuracy 82.3% vs. Router-R1 66.5%, with the strongest agent used for under 25% of calls.
• Removing Lagrangian multipliers in System 1 drops average accuracy from 80.5% to 56.5% and routes all queries to the strongest agent.

Surprising findings

• Without Lagrangian penalties the controller routes all queries to the strongest agent, showing collapse to the most expensive model.
• Controller verification recall on incorrect responses is about 77-81%, with only 199-216 false rejections.

Limitations

• Experiments limited to math reasoning with verifiable correctness rewards.
• Multi-turn design adds inference latency and compute overhead versus one-shot routing.
• Lagrange multipliers are tuned as hyperparameters; relaxation is not strictly equivalent to the constrained problem.

Experimental setup

Datasets: MATH-lighteval

Benchmarks: MATH-lighteval MATH-500 Omni-MATH MinervaMath AGI-Eval-MATH AMC23 AIME24

Metrics: accuracy

Baselines: Router-R1 RouterDC RoBERTa Router Random Router Controller V1 Controller V2

Builds on: GRPO Router-R1 RouterDC Search-R1

Closest papers by meaning

• Reinforce LLM Reasoning through Multi-Agent Reflection (ICML 2025 · similarity 0.90)
• Selective Critique for Cost-Aware LLM Agents in Long-Horizon Decision Making (NeurIPS 2026 · similarity 0.88)
• From Debate to Equilibrium: Belief‑Driven Multi‑Agent LLM Reasoning via Bayesian Nash Equilibrium (ICML 2025 · similarity 0.88)
• $\texttt{Multi}^2$: Hierarchical Multi-Agent Decision-Making with LLM-Based Agents in Interactive Environments (ICML 2026 · similarity 0.88)
• HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems (ICML 2026 · similarity 0.88)
• Agentic Multi-Turn Reasoning: A Fairness Approach (NeurIPS 2026 · similarity 0.87)