Research map / Large language models / Reasoning and chain of thought

VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains Qwen2.5-VL with pure RL (GRPO plus Selective Sample Replay and Forced Rethinking triggers) to elicit self-reflective slow-thinking for multimodal reasoning, reaching open-source SoTA on several math and multi-discipline benchmarks.

Problem

Vision-language models lack slow-thinking, self-reflective reasoning comparable to text-only reasoning models; RL on multimodal data often fails to induce reflection and GRPO suffers from vanishing advantages.

Approach

Combines GRPO with Selective Sample Replay (filters zero-advantage groups and replays high-|advantage| samples from a buffer) and Forced Rethinking (appends rethinking trigger tokens to rollouts, keeps correct rethinking trajectories, adds an SFT loss), trained directly with RL without distillation.

Key results

• VL-Rethinker-72B reaches 80.4% on MathVista and 63.5% on MathVerse, surpassing OpenAI-o1 (73.9% and 57.0%).
• VL-Rethinker-7B reaches 74.9% MathVista and 54.2% MathVerse, outperforming open 7B reasoning models such as OpenVLThinker (70.2%, 47.9%).
• Ablation: removing Forced Rethinking drops MathVision from 32.3 to 29.8 for the 7B model; removing SSR and filtering (plain GRPO) drops it to 26.0.

Measured effects reported in the paper

Surprising findings

• Qwen2.5-VL-72B's fraction of non-zero-advantage examples falls from about 40% to below 20% during RL, showing vanishing advantages in multimodal GRPO.
• Multimodal RL does not naturally induce longer reasoning or wait-style reflection, unlike math-only RL, motivating forced rethinking.

Limitations

• Performance still lags human experts and OpenAI-o1 on general multimodal benchmarks such as EMMA and MEGA-Bench, attributed to limited high-quality multimodal training data.
• Qwen2.5-VL-32B-based and 72B results on MEGA-Bench are weaker for the 32B model (19.9 vs 13.3 base is improved but far below 72B), and the reflection behavior is not consistently induced by plain RL.

Experimental setup

Datasets: publicly available multimodal reasoning datasets novel web-collected queries

Benchmarks: MathVista MathVerse MathVision MMMU-Pro MMMU EMMA MEGA-Bench

Metrics: Pass@1 accuracy (greedy decoding)

Baselines: OpenAI-o1 GPT-4o Claude-3.5-Sonnet Gemini-2.0-Flash Qwen2.5-VL-72B QvQ-72B InternVL-2.5-78B OpenVLThinker R1-OneVision MM-Eureka

Builds on: GRPO DeepSeek-R1 PPO Prioritized Experience Replay DAPO S1 budget forcing

Closest papers by meaning

• Semi-off-Policy Reinforcement Learning for Vision-Language Slow-Thinking Reasoning (NeurIPS 2025 · similarity 0.91)
• Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.91)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.90)
• All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models (CVPR 2026 · similarity 0.90)
• ProxyThinker: Test-Time Guidance through Small Visual Reasoners (ICLR 2026 · similarity 0.90)
• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)