AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

Tournament Style RL: Stabilizing Policy Optimization on Non Verifiable Problems

Gurusha Juneja, Shubham Milind Phal, Jennifer She, Lisa Wang, Dorsa Sadigh, Anca Dragan, William Yang Wang

ICML 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use win-rate against 4 fixed anchors instead of a single scalar rubric score as rewardGRPO on LLaMA-3.1-8B · Storieswin-rate (%): 47.5 -> 74.0Improved
use multiple anchors instead of a single anchor comparison (ablation)TSRL · Storieswin-rate (%): 13.5 -> 74.0Improved
increase number of anchors k from 1 to 10 (ablation)TSRL · all four taskswin-rate: average of 52 win-rate points from k = 1 to k = 10Improved
use diverse anchors instead of low-diversity anchors (ablation)TSRL · Stories (LLaMA-3.1-8B)win-rate (%): 12.8 -> 74.0Improved
replace Kimi-K2 judge with a weak LLaMA-3.1-8B judge (ablation)TSRL · Stories (LLaMA-3.1-8B)win-rate (%): 74.0 -> 70.0Hurt
replace Kimi-K2 judge with a weak judge (ablation)Rubric-based RL · Stories (LLaMA-3.1-8B)win-rate (%): 47.5 -> 34.0Hurt
use higher-quality anchors (Flash -> Pro -> human) (ablation)TSRL · Stories (LLaMA-3.1-8B)win-rate (%): 65.5 -> 70.0 -> 74.0Improved
replace fixed anchors with in-batch ranking reward (ablation)TSRL · Stories (LLaMA-3.1-8B)win-rate (%): 74.0 -> 55.0Hurt
use online win-rate reward instead of offline DPO on the same anchor comparisonsLLaMA-3.1-8B · Storieswin-rate (%): 57.5 -> 74.0Improved
refresh anchors from the current policy every 50 steps (self-bootstrap) (ablation)TSRL · Stories (LLaMA-3.1-8B)win-rate (%): 74.0 -> 81.5Improved
scale policy size from Qwen2.5-3B to 7B (ablation)TSRL · Storieswin-rate (%): gap over rubric 14.2 -> 28.0Improved
use ordered partial-correctness anchors instead of binary correctness rewardLLaMA-3.1-8B · MATHaccuracy: 50.0 -> 54.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy