Research map / Vision-language and multimodal / Multimodal LLMs

To Think or Not To Think: A Study of Thinking in Rule-Based Visual Reinforcement Fine-Tuning

NeurIPS 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies whether explicit thinking is needed in rule-based RFT (GRPO) for MLLMs, proposing No-Thinking-RFT with an equality accuracy reward plus Think-After-Answer and Adaptive-Thinking variants, tested on classification and visual reasoning with Qwen2-VL 2B/7B.

Problem

Whether explicit chain-of-thought thinking is necessary or beneficial during rule-based reinforcement fine-tuning of multimodal LLMs, given its high computational cost and possible hindrance to reward convergence.

Approach

Removes thinking from RFT by using a direct-answer prompt and a strict equality accuracy reward without format reward (No-Thinking-RFT); further tests placing reasoning after the answer (Think-After-Answer) and letting the model choose whether to think (Adaptive-Thinking).

Key results

• On 2B few-shot classification, No-Thinking-RFT beats Thinking-RFT on 10 of 11 datasets, with 3.14% higher average accuracy.
• On CVBench (2B), No-Thinking-RFT reaches 76.76 overall vs 70.36 for Thinking-RFT, with fine-tuning time 139 min vs 599 min.
• Thinking-RFT takes 1587 min to fine-tune on ImageNet versus 94 min for No-Thinking-RFT and 35 min for SFT.

Measured effects reported in the paper

Surprising findings

• Visual perception and puzzle tasks do not need explicit thinking; No-Thinking-RFT matches or outperforms Thinking-RFT across model sizes.
• For inconsistent responses in Thinking-RFT, answer-tag accuracy and thinking-tag accuracy are both below overall accuracy.

Limitations

• Experiments limited to models of 7B parameters or smaller, so findings for larger models are unverified.
• Adaptive-Thinking converges to one response mode per task rather than adapting per problem.
• Inconsistency analysis relies on GPT-4o extraction, and strategies to improve consistency are left to future work.

Experimental setup

Datasets: ImageNet Caltech101 OxfordPets StanfordCars Flowers102 Food101 FGVCAircraft SUN397 DTD EuroSAT UCF101 SAT

Benchmarks: CVBench MathVista MathVision PuzzleVQA AlgoPuzzleVQA 11-dataset few-shot classification base-to-new classification

Metrics: accuracy harmonic mean accuracy fine-tuning time (min) inference time (min)

Baselines: Thinking-RFT SFT Think-After-Answer Qwen2-VL-2B-Instruct zero-shot offline DPO

Builds on: GRPO DeepSeek-R1 Thinking-RFT CoOp

Closest papers by meaning

• On the Faithfulness of Visual Thinking: Measurement and Enhancement (NeurIPS 2026 · similarity 0.91)
• Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models (NeurIPS 2025 · similarity 0.90)
• When to Think and When to Look: Uncertainty-Guided Lookback (CVPR 2026 · similarity 0.89)
• On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs (ICML 2026 · similarity 0.89)
• Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.88)
• Thinking with Programming Vision: Towards a Unified View for Thinking with Images (CVPR 2026 · similarity 0.88)