Research map / Multi-agent systems and games / LLM agents

Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AT-GRPO, an agent- and turn-wise grouped RL algorithm plus a training system for on-policy RL over multi-agent LLM systems, evaluated on game, planning, coding, and math with Qwen3 1.7B and 8B.

Problem

Applying on-policy RL to multi-agent LLM systems (MAS) is underexplored: standard GRPO grouping assumes identical prompts, which fails when prompts differ by role and turn, and existing RL frameworks support only single-model training.

Approach

Group rollouts per agent and per turn using tree-structured sampling (K branches from the same state, greedy execution of the best branch), add agent-wise mixed team/local rewards, and train either a shared or role-specialized policy through a multi-model training system with per-model resource pools.

Key results

• On Plan-Path with Qwen3-1.7B, MAS + AT-GRPO (per-role policies) reaches 97.00% vs 5.00% single-agent baseline; Sokoban with Qwen3-8B reaches 98.00%.
• On AIME24 with Qwen3-8B, per-role AT-GRPO achieves 57.00% vs 18.30% single agent (+38.70).
• Naive MAS + GRPO degrades on some tasks, e.g., Qwen3-8B CodeContests drops from 17.60 (MAS) to 10.30.

Surprising findings

• Naive GRPO applied to MAS often degrades performance compared to prompt-only MAS, attributed to heterogeneous interaction histories breaking the identical-state assumption.
• Swapping the two trained role-specific policies drops Plan-Path accuracy from 96.0% to 6.0%, showing role-specific non-interchangeability.

Limitations

• Greedy selection of best-of-K branch is justified by an assumption (monotonicity of verifiable reward) that is strong and not verified empirically.
• Gains on math and code are modest relative to planning and sometimes small (e.g., +2.35 on CodeContests for 8B), and the base models are likely saturated on these domains.

Experimental setup

Datasets: Polaris-Dataset-53K APPS CodeContests SymBench gsm8k

Benchmarks: Sudoku Sokoban Plan-Path LiveCodeBench APPS CodeContests AIME24 AIME25 OlympiadBench

Metrics: accuracy pass@1 gain over single agent

Baselines: Single agent Single agent + GRPO MAS (prompt-only) MAS + GRPO MAPoRL MARFT CURE

Builds on: GRPO GiGPO VERL HybridFlow

Closest papers by meaning

• Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025 · similarity 0.93)
• Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems (NeurIPS 2026 · similarity 0.93)
• MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systems (ICML 2026 · similarity 0.91)
• Agentic Reinforced Policy Optimization (ICLR 2026 · similarity 0.90)
• Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
• Accelerating RL for LLM Reasoning with Optimal Advantage Regression (NeurIPS 2025 · similarity 0.89)