AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

URPO: A Unified Reward & Policy Optimization Framework for Large Language Models

Songshuo Lu, Hua Wang, Zhi Chen, Yaohua Tang

AAAI 2026 · Reinforcement learning › RL from feedback · approach: GRPO stability and variants · 12 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace separate RM-gen reward with unified URPOGRPO on Qwen2.5-7B Base · AlpacaEval 2.0win rate: 42.24 -> 44.84Improved
replace separate RM-gen reward with unified URPOGRPO on Qwen2.5-7B Base · AIME 24&25 (mean@8)accuracy: 12.08 -> 16.46Improved
replace separate RM-gen reward with unified URPOGRPO on Qwen2.5-7B+SFT · Math average (SFT start)average accuracy: 31.64 -> 31.83Improved
replace separate RM-gen reward with unified URPOGRPO on Qwen2.5-7B+SFT · GSM8K (SFT start)accuracy: 90.37 -> 87.79Hurt
replace RM-gen reward with URPOGRPO on Qwen3-8B-Base · Math averageaverage accuracy: 39.75 -> 39.57No clear effect
replace RM-gen reward with URPOGRPO on OctoThinker Llama3.1-8B-Base · AlpacaEval 2.0win rate: 36.15 -> 41.49Improved
add instruction data to the P+R mix (1:1:0 -> 1:1:1) (ablation)URPO · AlpacaEval 2.0win rate: 31.43 -> 44.84Improved
add reasoning data to the P+I mix (1:0:1 -> 1:1:1) (ablation)URPO · Math averageaverage accuracy: 28.58 -> 35.66Improved
add reasoning data to preference data (ablation)URPO · RewardBenchmean score: 83.55 -> 85.15Improved
add preference data to the R+I mix (0:1:1 -> 1:1:1 style) (ablation)URPO · RewardBenchmean score: 62.39 -> 85.15Improved
double preference ratio (1:1:0 -> 2:1:0) (ablation)URPO · RewardBenchmean score: 85.15 -> 85.31No clear effect
raise preference ratio (1:1:1 -> 3:1:1) (ablation)URPO · Math average / AlpacaEvalmetric: Math 35.66 -> 35.77; AlpacaEval 44.84 -> 42.11Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy