Research map / Multi-agent systems and games / LLM agents
MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference
Kaiyan Zhang, Kai Tian, Runze Liu, Sihang Zeng, Xuekai Zhu, Guoli Jia, Yuchen Fan, Xingtai Lv, Yuxin Zuo, Che Jiang, Yuru wang, Jianyu Wang and 9 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents MARTI, an open-source framework for scalable reinforced training and inference of multi-agent LLM systems, with centralized multi-agent interaction, distributed policy training, and multi-turn asynchronous rollouts.
Problem
Training and running multi-agent LLM systems with reinforcement learning is hard to scale and efficiently orchestrate.
Approach
An open-source framework combining centralized multi-agent interaction with distributed policy training and multi-turn asynchronous rollouts, plus dynamic workflows for multi-agent interaction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reward shaping (historical-performance delta) (ablation) | MAD 2x2, Qwen2.5-3B · avg of AIME, AMC, MATH500 | accuracy: 36.6 -> 45.6 | Improved |
| add reward shaping (historical-performance delta) (ablation) | MoA 3x1, Qwen2.5-3B · avg of AIME, AMC, MATH500 | accuracy: 38.1 -> 43.1 | Improved |
| train multi-agent debate with RL instead of untrained MAD | Llama-3.2-3B-Instruct · avg of AIME, AMC, MATH500 | accuracy: 19.5 -> 32.1 | Improved |
| use multi-agent RL (MAD 2x2) instead of single-agent RL | Llama-3.2-3B-Instruct · avg of AIME, AMC, MATH500 | accuracy: 28.7 -> 32.1 | Improved |
| use untrained multi-agent workflows instead of majority voting | Qwen and reasoning models · benchmarks under equal budget | accuracy | Hurt |
| use GRPO instead of REINFORCE++ (ablation) | MAD 2x2, Qwen2.5-3B · avg of AIME, AMC, MATH500 | accuracy: 45.6 -> 46.0 | No clear effect |
| use GRPO instead of REINFORCE++ (ablation) | Single-agent, Qwen2.5-3B · avg of AIME, AMC, MATH500 | accuracy: 37.6 -> 37.9 | No clear effect |
| use multi-agent debate with TTRL instead of single-agent | DeepScaleR-1.5B-Preview · AIME | score: 53.5 -> 65.0 | Improved |
| use asynchronous rollouts (x64) instead of synchronous (ablation) | MAD, 8 interaction rounds · rollout time | seconds: 3308.0 -> 2928.0 | Improved |
| use asynchronous rollouts (x32) instead of synchronous (ablation) | MAD · rollout time | seconds: 593.5 -> 732.9 | Hurt |
| use asynchronous rollouts (x64) for a shallow 2-round workflow (ablation) | MAD · rollout time | seconds: 916.6 -> 887.5 | No clear effect |
Closest papers by meaning
- Reinforce LLM Reasoning through Multi-Agent Reflection (ICML 2025 · similarity 0.91)
- Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems (NeurIPS 2026 · similarity 0.91)
- MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems (ICML 2025 · similarity 0.90)
- $\texttt{Multi}^2$: Hierarchical Multi-Agent Decision-Making with LLM-Based Agents in Interactive Environments (ICML 2026 · similarity 0.89)
- Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation (ICLR 2026 · similarity 0.89)
- From Debate to Equilibrium: Belief‑Driven Multi‑Agent LLM Reasoning via Bayesian Nash Equilibrium (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
