Research map / Large language models / Reasoning and chain of thought
Learning Consistency in Reward Modeling for Multi-Modal Reasoning
Chen Li, Bolin Ni, Boxin Zhang, Ke Ye, Jinnian Zhang, Houwen Peng, Han Hu, Nanning Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use soft continuous reward instead of hard reward | CRM-RL-7B · MathVista | accuracy: 74.3% -> 75.2% | Improved |
| use soft reward instead of hard reward (Qwen3-VL-8B) | CRM-RL-8B · Avg of 4 math benchmarks | Avg accuracy: 67.6% -> 68.5% | Improved |
| replace rule-based reward with CRM reward | Qwen2.5-VL-7B RL · MathVista and MathVision | accuracy: +2.8% on MathVista, +2.0% on MathVision | Improved |
| replace CompassVerifier reward with CRM hard reward | Qwen2.5-VL-7B RL · Avg of 4 math benchmarks | Avg accuracy: 56.8% -> 57.4% | Improved |
| add chain-of-thought to CRM (hard reward) (ablation) | CRM-RL-7B · MathVista | accuracy: 73.4% -> 74.3% | Improved |
| add chain-of-thought to CRM (soft reward) (ablation) | CRM-RL-7B · MathVista | accuracy: 73.1% -> 75.2% | Improved |
| add chain-of-thought to CRM (soft reward) (ablation) | CRM-RL-7B · MathVision | accuracy: 28.8% -> 30.6% | Improved |
| add CoT to CRM soft reward judging (ablation) | CRM soft reward · 50 hard cases | Reward accuracy: 58.0% -> 72.0% | Improved |
| add CoT in static judging (ablation) | CRM · Consistency validation set | Val accuracy: 97.2% -> 98.0% | Improved |
| add image input to the consistency reward model (ablation) | CRM · Consistency validation set | Val accuracy: 98.0% -> 96.7% | Hurt |
| scale reward model from 1.5B to 7B (ablation) | CRM · Win rate vs VLMEvalKit (Avg) | Win rate: 64.5% -> 73.5% | Improved |
| scale reward model from 1.5B to 7B (ablation) | CRM · RL test accuracy (Avg) | accuracy: 51.9% -> 52.6% | Improved |
Closest papers by meaning
- RM-R1: Reward Modeling as Reasoning (ICLR 2026 · similarity 0.91)
- Enhancing the Outcome Reward-based RL Training of MLLMs with Self-Consistency Sampling (NeurIPS 2025 · similarity 0.91)
- Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
- The Trickle-down Impact of Reward Inconsistency on RLHF (ICLR 2024 · similarity 0.89)
- VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Models (ICLR 2026 · similarity 0.89)
- Self-Aligned Reward: Towards Effective and Efficient Reasoners (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
