Research map / Vision-language and multimodal / Multimodal LLMs

Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning

NeurIPS 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FAST-GRPO, a GRPO variant for LVLM reasoning that uses difficulty estimates (extrinsic pass-rate plus image complexity) to adaptively shape length rewards and KL regularization, yielding shorter yet more accurate fast/slow reasoning on seven math/multimodal benchmarks.

Problem

RL-trained slow-thinking LVLMs either fail to scale reasoning length usefully (RL-only) or produce verbose outputs with marginal accuracy gains (SFT-RL), with no effective balance of fast and slow thinking for visual reasoning.

Approach

Estimate question difficulty online (pass-rate combined with GLCM and ViT-entropy image complexity), then use a difficulty-aware length reward (shorten correct answers on easy questions, lengthen incorrect ones on hard questions) and a difficulty-dependent KL coefficient within GRPO, with slow-to-fast curriculum sampling.

Key results

• FAST-7B reaches 73.8 on MathVista and 50.6 on MathVerse, with average accuracy 59.0 vs 55.7 for the Kimi length-penalty baseline across five benchmarks, and average length 162.2 tokens.
• Against Qwen2.5-VL-7B, FAST-7B improves average accuracy by over 10% relative while cutting length (e.g., MathVista 189.1 to 120.7 tokens).
• On Geometry3K, FAST-7B reaches 46.4% overall accuracy vs 40.3% for R1-OneVision, using 239 vs 731 tokens.

Measured effects reported in the paper

Surprising findings

• Removing KL regularization (beta=0) boosts hard-question accuracy substantially but risks forgetting on easy questions, showing no static beta is optimal.
• Naive GRPO overfits: validation accuracy peaks around step 150 and then declines while training accuracy keeps rising, with reasoning chains becoming inconsistent with final answers.

Limitations

• Only evaluated up to 32B; behavior at 70B+ scale untested.
• Image complexity metric shows only moderate human correlation (SRCC 0.49).
• 32B model trained for only 3 epochs, likely sub-optimal checkpoint.

Experimental setup

Datasets: LLaVA-CoT Mulberry MathV-360K Geometry3K

Benchmarks: MathVision MathVerse MathVista MM-Math WeMath DynaMath MM-Vet MM-K12 Bingo MMHal MMVP MMEval-Pro

Metrics: accuracy response length (tokens)

Baselines: R1-OneVision OpenVLThinker MM-R1 LMM-R1 MM-Eureka Vision-R1 Curr-ReFT Mulberry Kimi length penalty DAST CosFn

Builds on: GRPO DeepSeek-R1 Kimi 1.5 length penalty DAST

Closest papers by meaning

• Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.92)
• VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Semi-off-Policy Reinforcement Learning for Vision-Language Slow-Thinking Reasoning (NeurIPS 2025 · similarity 0.90)
• All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models (CVPR 2026 · similarity 0.90)
• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)
• S-GRPO: Early Exit via Reinforcement Learning in Reasoning Models (NeurIPS 2025 · similarity 0.90)