AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference

Kaiyan Zhang, Kai Tian, Runze Liu, Sihang Zeng, Xuekai Zhu, Guoli Jia, Yuchen Fan, Xingtai Lv, Yuxin Zuo, Che Jiang, Yuru wang, Jianyu Wang and 9 more

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Presents MARTI, an open-source framework for scalable reinforced training and inference of multi-agent LLM systems, with centralized multi-agent interaction, distributed policy training, and multi-turn asynchronous rollouts.

Problem

Training and running multi-agent LLM systems with reinforcement learning is hard to scale and efficiently orchestrate.

Approach

An open-source framework combining centralized multi-agent interaction with distributed policy training and multi-turn asynchronous rollouts, plus dynamic workflows for multi-agent interaction.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add reward shaping (historical-performance delta) (ablation)MAD 2x2, Qwen2.5-3B · avg of AIME, AMC, MATH500accuracy: 36.6 -> 45.6Improved
add reward shaping (historical-performance delta) (ablation)MoA 3x1, Qwen2.5-3B · avg of AIME, AMC, MATH500accuracy: 38.1 -> 43.1Improved
train multi-agent debate with RL instead of untrained MADLlama-3.2-3B-Instruct · avg of AIME, AMC, MATH500accuracy: 19.5 -> 32.1Improved
use multi-agent RL (MAD 2x2) instead of single-agent RLLlama-3.2-3B-Instruct · avg of AIME, AMC, MATH500accuracy: 28.7 -> 32.1Improved
use untrained multi-agent workflows instead of majority votingQwen and reasoning models · benchmarks under equal budgetaccuracyHurt
use GRPO instead of REINFORCE++ (ablation)MAD 2x2, Qwen2.5-3B · avg of AIME, AMC, MATH500accuracy: 45.6 -> 46.0No clear effect
use GRPO instead of REINFORCE++ (ablation)Single-agent, Qwen2.5-3B · avg of AIME, AMC, MATH500accuracy: 37.6 -> 37.9No clear effect
use multi-agent debate with TTRL instead of single-agentDeepScaleR-1.5B-Preview · AIMEscore: 53.5 -> 65.0Improved
use asynchronous rollouts (x64) instead of synchronous (ablation)MAD, 8 interaction rounds · rollout timeseconds: 3308.0 -> 2928.0Improved
use asynchronous rollouts (x32) instead of synchronous (ablation)MAD · rollout timeseconds: 593.5 -> 732.9Hurt
use asynchronous rollouts (x64) for a shallow 2-round workflow (ablation)MAD · rollout timeseconds: 916.6 -> 887.5No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy