AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking

Qiang Zhang, Boli Chen, Fanrui Zhang, Ruixue Ding, Shihang Wang, Qiuchen Wang, Yinfeng Huang, Haonan Zhang, Rongxiang Zhu, Xin Li, Houquan Zhou, Pengjun Xie and 3 more

ICML 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace pointwise rewards (GRPO) with tournament-based relative rankingQwen3-8B-Base · Open-Travelmean win rate: 16.4 -> 41.8Improved
replace pointwise rewards with ArenaRLQwen3-8B-Base · Open-DeepResearchmean win rate: 25.2 -> 64.3Improved
replace pointwise rewards with ArenaRLQwen3-8B-Base · Writing benchmarksmean score: 73.0 -> 80.4Improved
use seeded single-elimination instead of round-robin (ablation)ArenaRL · Open-Travelmean win rate: 32.9 -> 32.5No clear effect
use seeded single-elimination instead of Swiss system (ablation)ArenaRL · Open-Travelmean win rate: 28.3 -> 32.5Improved
add greedy-decoded anchor seeding (ablation)ArenaRL · Open-Travelmean win rate: 37.9 -> 41.8Improved
add seed-based pre-ranking (ablation)ArenaRL · Open-Travelmean win rate: 33.3 -> 41.8Improved
evaluate intermediate reasoning trajectory, not only final answer (ablation)ArenaRL · Open-Travelmean win rate: 29.6 -> 41.8Improved
add bidirectional pairwise evaluation (ablation)ArenaRL · Open-Travelmean win rate: 36.1 -> 41.8Improved
increase group size N from 2 to 16 (ablation)ArenaRL · Open-Travelmean win rate: 20.8 -> 41.8Improved
use stronger judge (Claude-4-Sonnet vs GPT-4.1) (ablation)ArenaRL · Open-Travelmean win rate: 37.9 -> 43.2Improved
train ArenaRL directly without cold start (ablation)Qwen3-8B · Open-Travel Searchwin rate: 0 -> 71%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy