Research map / Vision-language and multimodal / Multimodal LLMs

Let VLMs Grade Their Own Thoughts: A Self-Quantification Approach to Reasoning-Aware Reward Modeling

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Video-RAISE, an RL post-training method for video VLMs whose rewards come from the model's own answer-token probabilities (SCRE for strict-logic tasks, IGSR for open-ended tasks), reporting SOTA on six video benchmarks at 7B.

Problem

Existing RL post-training for video reasoning in VLMs relies on external, human-defined reasoning constraints, which may limit the model's discovery of effective reasoning paths and can hurt performance.

Approach

Replace sparse rule-based 0/1 rewards with continuous rewards derived from the model's intrinsic answer-token confidence: SCRE multiplies position-decayed answer token probabilities and penalizes variance, while IGSR groups semantically similar candidates and re-ranks them within each group by average negative log-probability.

Key results

• Video-RAISE-7B reaches 52.8% on VideoMMMU at 16 frames, +3.0 over Video-R1-7B (49.8%).
• Video-RAISE-7B reaches 53.0% on VideoMMMU at 32 frames vs 50.4% for CARE-7B, and 60.7 on VideoMME at 32 frames vs 59.6.
• Reasoning path consistency (Match) of nearly 90% across benchmarks, roughly double Qwen2.5-VL-Instruct on several benchmarks.

Surprising findings

• Sequence-level average confidence overlaps heavily between correct and incorrect answers, while answer-token-level confidence polarizes incorrect answers toward both 0.0 and 1.0.
• The GRPO baseline shows reasoning degradation on several benchmarks, with lower answer correctness than the base model on TempCompass.

Limitations

• Reliance on the model's own confidence assumes calibration; confidently wrong answers still receive high reward.
• Gains over prior RL methods are modest (often 1-3 points) and evaluation is limited to 7B video models.

Experimental setup

Datasets: Video-R1-260k Video-R1-CoT-165k

Benchmarks: VSI-Bench VideoMMMU MMVU MVBench TempCompass VideoMME

Metrics: accuracy answer correctness reasoning path consistency (match rate)

Baselines: Video-R1-7B CARE-7B Qwen2.5-VL Qwen2.5-VL-SFT Qwen2.5-VL-GRPO VideoTree Video-RTS-7B ReAgent-V RRPO-7B GPT-4o

Builds on: GRPO Video-R1 GRPO-CARE

Closest papers by meaning

• Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards (NeurIPS 2025 · similarity 0.91)
• Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling (NeurIPS 2026 · similarity 0.91)
• When Thinking Drifts: Evidential Grounding for Robust Video Reasoning (NeurIPS 2025 · similarity 0.91)
• Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.91)
• Latent Chain-of-Thought for Visual Reasoning (NeurIPS 2025 · similarity 0.90)
• SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning (ICML 2026 · similarity 0.90)