AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

Breaking the Self-Confirming Loop: Diagnosing and Mitigating Systemic Reward Bias in Self-Rewarding RL

Chuyi Tan, Peiwen Yuan, Xinglin Wang, Yiwei Li, Shaoxiong Feng, Yueqi Zhang, Jiayi Shi, Ji Zhang, Boyuan Pan, Yao Hu, Kan Li

ICML 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add ensemble of sub-policies (k=2) instead of single-policy rewarding (ablation)RLER (Interpolation v3) · Arithmetic test set (500 problems)Avg@16: 63.2 -> 69.6Improved
add adaptive reward interpolation to ensemble SC (ablation)RLER · Arithmetic test set (500 problems)Avg@16: 67.6 -> 69.6Improved
add disagreement-aware rollout selection (ablation)RLER · Arithmetic test set (500 problems)Avg@16: 69.6 -> 71.5Improved
replace fixed alpha=0.5 with full adaptive alpha(x) in RLER (ablation)RLER · DAPO-Math-17K, six benchmarksAvg@8: 36.2 -> 37.5Improved
use full adaptive alpha(x) instead of annealed schedule (ablation)single-policy interpolation · DAPO-Math-17K, six benchmarksAvg@8: 33.6 -> 36.0Improved
increase ensemble size from k=2 to k=4 at fixed budget 16 (ablation)RLER · DAPO-Math-17K, six benchmarksAvg@8: 37.5 -> 37.6No clear effect
increase ensemble size from k=4 to k=8 at fixed budget 16 (ablation)RLER · DAPO-Math-17K, six benchmarksAvg@8: 37.6 -> 37.2Hurt
raise total rollout budget from 16 to 32 with k=4 (ablation)RLER · DAPO-Math-17K, six benchmarksAvg@8: 37.6 -> 37.9Improved
vary interpolation bounds beta- and beta+ (ablation)RLER · DAPO-Math-17K, six benchmarksAvg@8: 36.9 to 37.5No clear effect
use RLER instead of SC at temperature 0.5Llama-3.2-3B-Instruct arithmetic · Arithmetic test setcheckpoint accuracy variance: 78.92 -> 4.26Improved
use RLER instead of SC at temperature 0.9Qwen2.5-Math-7B · DAPO-Math-17KPass@1: 33.0 -> 37.5Improved
use disagreement-aware selection instead of selecting all rollouts when ensemble majority is wrong (ablation)RLER · DAPO-Math-17Kreward noise rho_noise: 65.5 -> 50.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy