Research map / Reinforcement learning / RL from feedback
ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking
Qiang Zhang, Boli Chen, Fanrui Zhang, Ruixue Ding, Shihang Wang, Qiuchen Wang, Yinfeng Huang, Haonan Zhang, Rongxiang Zhu, Xin Li, Houquan Zhou, Pengjun Xie and 3 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace pointwise rewards (GRPO) with tournament-based relative ranking | Qwen3-8B-Base · Open-Travel | mean win rate: 16.4 -> 41.8 | Improved |
| replace pointwise rewards with ArenaRL | Qwen3-8B-Base · Open-DeepResearch | mean win rate: 25.2 -> 64.3 | Improved |
| replace pointwise rewards with ArenaRL | Qwen3-8B-Base · Writing benchmarks | mean score: 73.0 -> 80.4 | Improved |
| use seeded single-elimination instead of round-robin (ablation) | ArenaRL · Open-Travel | mean win rate: 32.9 -> 32.5 | No clear effect |
| use seeded single-elimination instead of Swiss system (ablation) | ArenaRL · Open-Travel | mean win rate: 28.3 -> 32.5 | Improved |
| add greedy-decoded anchor seeding (ablation) | ArenaRL · Open-Travel | mean win rate: 37.9 -> 41.8 | Improved |
| add seed-based pre-ranking (ablation) | ArenaRL · Open-Travel | mean win rate: 33.3 -> 41.8 | Improved |
| evaluate intermediate reasoning trajectory, not only final answer (ablation) | ArenaRL · Open-Travel | mean win rate: 29.6 -> 41.8 | Improved |
| add bidirectional pairwise evaluation (ablation) | ArenaRL · Open-Travel | mean win rate: 36.1 -> 41.8 | Improved |
| increase group size N from 2 to 16 (ablation) | ArenaRL · Open-Travel | mean win rate: 20.8 -> 41.8 | Improved |
| use stronger judge (Claude-4-Sonnet vs GPT-4.1) (ablation) | ArenaRL · Open-Travel | mean win rate: 37.9 -> 43.2 | Improved |
| train ArenaRL directly without cold start (ablation) | Qwen3-8B · Open-Travel Search | win rate: 0 -> 71% | Improved |
Closest papers by meaning
- Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025 · similarity 0.89)
- Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
- Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs (ICLR 2026 · similarity 0.87)
- Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation (NeurIPS 2026 · similarity 0.87)
- Online Matching with Stochastic Rewards: Provable Better Bound via Adversarial Reinforcement Learning (ICML 2024 · similarity 0.87)
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
