AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning

YiFan Zhang, Xingyu Lu, Xiao Hu, Chaoyou Fu, Bin Wen, Tianke Zhang, Changyi Liu, Kaiyu Jiang, Kaibing Chen, Kaiyu Tang, Haojie Ding, Jiankang Chen and 6 more

ICLR 2026 · Reinforcement learning › RL from feedback · approach: RL for LLM reasoning · 66 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with StableReinforce RL after Long-CoT SFT instead of SFT only (ablation)R1-Reward · VL-Reward BenchOverall Acc: 64.80 -> 71.92Improved
add advantage filter (3-sigma) (ablation)StableReinforce · VL-Reward BenchOverall Acc: 68.96 -> 71.92Improved
add pre-clip of log ratio (ablation)StableReinforce · VL-Reward BenchOverall Acc: 67.36 -> 71.92Improved
use StableReinforce instead of plain Reinforce++ (ablation)R1-Reward · VL-Reward BenchOverall Acc: Collapse -> 71.92Improved
RL-based reward modeling instead of MLP reward head on same 200K data (ablation)Qwen2.5-VL-7B · VL-Reward BenchOverall Acc: 56.41 -> 71.92Improved
use R1-Reward SFT pairwise comparison vs MM-RLHF-Reward critic-then-score on same data (ablation)Qwen2.5-VL-7B · VL-Reward BenchOverall Acc: 60.80 -> 64.80Improved
majority voting over 15 samplesR1-Reward · VL-Reward BenchOverall Acc: 71.92 -> 76.46Improved
majority voting over 15 samplesR1-Reward · MM-RLHF-Reward BenchAcc+: 54.35 -> 67.39Improved
majority voting over 15 samplesR1-Reward · Multimodal RewardBenchOverall: 82.2 -> 83.3Improved
R1-Reward vs MM-RLHF-Reward on in-distribution benchmarkR1-Reward · MM-RLHF-Reward BenchAcc: 82.00 -> 80.59Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy