AI Paper Atlas

Research map / Multi-agent systems and games / Games and equilibria

SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning

Bo Liu, Simon Yu, Zichen Liu, Leon Guertler, Penghui Qi, Daniel Balcells, Mickel Liu, Cheston Tan, Weiyan Shi, Min Lin, Wee Sun Lee, Natasha Jaques

ICLR 2026 · Multi-agent systems and games › Games and equilibria · approach: Learning in Markov games · 107 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add multi-game SPIRAL self-play trainingQwen3-4B-Base · Reasoning suite average (8 benchmarks)average accuracy: 34.0 -> 44.5Improved
add multi-game SPIRAL self-play trainingQwen3-8B-Base · Reasoning suite average (8 benchmarks)average accuracy: 39.5 -> 49.6Improved
add SPIRAL to an already instruction-tuned modelLlama-3.1-8B-Instruct · Reasoning suite average (8 benchmarks)average accuracy: 23.9 -> 25.9Improved
add SPIRAL to a model already RLVR-trainedDeepSeek-Distill-Qwen-7B · Reasoning suite average (8 benchmarks)average accuracy: 60.4 -> 61.8Improved
replace SFT on expert game trajectories with SPIRAL self-play (ablation)Qwen3-4B-Base · Reasoning suite average (8 benchmarks)average accuracy: 39.7 -> 44.5Improved
scale SFT data from 25k to 52k trajectories (ablation)SFT-Multi on Qwen3-4B-Base · Reasoning suite average (8 benchmarks)average accuracy: 39.7 -> 39.7No clear effect
SFT hurts when applied to an RLVR-trained modelDeepSeek-Distill-Qwen-7B · Reasoning suite average (8 benchmarks)average accuracy: 60.4 -> 58.3Hurt
use self-play instead of fixed Mistral opponent (ablation)SPIRAL on Kuhn Poker · Reasoning suite average (8 benchmarks)average accuracy: 29.6 -> 43.4Improved
use self-play instead of fixed Gemini opponent (ablation)SPIRAL on Kuhn Poker · Reasoning suite average (8 benchmarks)average accuracy: 33.4 -> 43.4Improved
train on three games instead of one (add games) (ablation)SPIRAL on Qwen3-4B-Base · Reasoning suite average (8 benchmarks)average accuracy: 40.0 -> 44.5Improved
add Simple Negotiation to TicTacToe+KuhnPoker training (ablation)SPIRAL on Qwen3-4B-Base · Reasoning suite average (8 benchmarks)average accuracy: 41.3 -> 44.5Improved
add multi-game training over single-game specialists (ablation)SPIRAL game-play · Game win rate vs Gemini-2.0-Flash (average over 6 games)win rate (%): 52.9% -> 59.5%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy