Research map / Large language models / Reasoning and chain of thought
Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TON, a two-stage SFT-then-GRPO method where random "thought dropout" replaces reasoning with an empty think block, letting VLMs learn when to skip reasoning and cutting completion length by up to 90% without hurting accuracy.
Problem
GRPO-trained vision-language models always generate full reasoning traces regardless of task difficulty, inflating token usage and compute; it is unclear whether VLMs can learn to decide when reasoning is needed.
Approach
Thought dropout during SFT randomly replaces reasoning traces with empty thoughts to create a think-or-not format as a cold start; a subsequent GRPO stage with outcome rewards lets the policy explore and choose when to skip thinking, reducing length while keeping or improving accuracy.
Key results
- On CLEVR with Qwen-2.5-VL-3B, TON reaches 98.5% accuracy vs 93.5% for GRPO while cutting completion length from 227 to 28 tokens (-87%).
- On GeoQA with Qwen-2.5-VL-7B, TON improves accuracy from 44% to 61% while reducing completion length from 1069 to 112 tokens (-90%).
- On AITZ, TON reduces task-level thought length from about 3.6K to 0.9K tokens with comparable OOD exact accuracy to GRPO (0.59 vs 0.59 for 3B).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TON instead of vanilla GRPO | Qwen2.5-VL-3B · CLEVR | accuracy: 93.5 -> 98.5 | Helped |
| Use TON instead of vanilla GRPO | Qwen2.5-VL-3B · GeoQA | accuracy: 37 -> 51 | Helped |
| Use TON instead of vanilla GRPO | Qwen2.5-VL-7B · GeoQA | accuracy: 44 -> 61 | Helped |
| Use TON instead of vanilla GRPO | Qwen2.5-VL-3B · AITZ OOD | task-level output length: 3664 -> 922 | Helped |
| Use TON instead of vanilla GRPO | Qwen2.5-VL-7B · AITZ OOD | average exact accuracy: 0.23 -> 0.45 | Helped |
| Add reasoning traces during SFT (ablation) | Qwen2.5-VL-3B · AITZ agent dataset | test accuracy: 0.39 -> 0.67 | Helped |
| Add reasoning traces during GRPO (ablation) | Qwen2.5-VL-3B · AITZ agent dataset | test accuracy: 0.58 -> 0.74 | Helped |
| Use random thought dropout rather than difficulty-aware thought dropout (ablation) | TON · GeoQA | accuracy: 50.5% -> 51.0% | Helped |
| Vary SFT thought dropout probability among 20%, 50%, and 80% (ablation) | TON · AITZ | training reward | No clear effect |
| Replace thought-dropout SFT with a hybrid prompt encouraging direct answers (ablation) | GRPO · AITZ | completion length | No clear effect |
| Use hybrid prompting without thought-dropout SFT (ablation) | GRPO · GeoQA and AITZ | skip-thought outputs: 2 ‘skip’ cases in GeoQA and none in AITZ | Hurt |
Surprising findings
- Vanilla GRPO almost never produces skip-think outputs (only 2 on GeoQA, none on AITZ), so prompt-only hybrid thinking gives no length reduction.
- Omitting reasoning traces improved accuracy on GeoQA and CLEVR, a 'free-lunch' effect.
Limitations
- Experiments limited to 3B and 7B models; larger models not evaluated.
- Not evaluated on proprietary VLMs such as GPT-4o.
- Dropout is random rather than difficulty-aware; difficulty-aware dropout did not help.
Experimental setup
Datasets: CLEVR Super-CLEVR GeoQA AITZ GSM8K
Benchmarks: CLEVR Super-CLEVR GeoQA AITZ GSM8K
Metrics: accuracy step-level type accuracy step-level exact accuracy completion length training time
Baselines: GRPO Qwen-2.5-VL-3B SFT baseline hybrid thinking prompt
Builds on: GRPO DAPO Chain-of-Thought
Closest papers by meaning
- All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models (CVPR 2026 · similarity 0.94)
- Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning (NeurIPS 2025 · similarity 0.92)
- More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.92)
- Thinkless: LLM Learns When to Think (NeurIPS 2025 · similarity 0.91)
- VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Semi-off-Policy Reinforcement Learning for Vision-Language Slow-Thinking Reasoning (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.