Research map / Reinforcement learning / RL from feedback
R1-ShareVL: Incentivizing Reasoning Capabilities of Multimodal Large Language Models via Share-GRPO
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Share-GRPO, an RL method for MLLM reasoning that expands the question space via semantically consistent transformations and shares reasoning trajectories and reward information hierarchically across question variants, yielding R1-ShareVL-7B/32B.
Problem
GRPO applied to MLLMs suffers from sparse rewards (few correct reasoning paths, especially for weak models and hard questions) and advantage vanishing (homogeneous responses yield zero relative advantages).
Approach
Expand each question into semantically consistent variants (offline GPT-4o textual rewrites and online image transformations), let policy sample responses for every variant, share trajectories across variants, and compute advantages hierarchically: a global level over all variant responses plus a local level within each variant.
Key results
- R1-ShareVL-7B reaches 75.4 on MathVista versus 68.2 for the Qwen2.5-VL-7B base and 72.8 for GRPO, with average 61.2 across six benchmarks.
- R1-ShareVL-32B achieves average 67.2 versus 64.5 for Qwen2.5-VL-32B and +9.1 on MathVerse.
- On MathVista, Share-GRPO with 6+6 samples (75.4) surpasses GRPO with 24 samples (73.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train Qwen2.5-VL-7B with Share-GRPO | Qwen2.5-VL-7B · MathVista | accuracy: 68.2 -> 75.4 | Helped |
| Train Qwen2.5-VL-7B with Share-GRPO | Qwen2.5-VL-7B · MMStar | accuracy: 63.9 -> 67.0 | Helped |
| Train Qwen2.5-VL-7B with Share-GRPO | Qwen2.5-VL-7B · MMMU | accuracy: 58.6 -> 58.1 | No clear effect |
| Train Qwen2.5-VL-7B with Share-GRPO | Qwen2.5-VL-7B · MathVerse | accuracy: 49.2 -> 52.8 | Helped |
| Train Qwen2.5-VL-32B with Share-GRPO | Qwen2.5-VL-32B · MathVerse | accuracy: +9.1% | Helped |
| Add offline question variants with global shared advantage (ablation) | Share-GRPO on Qwen2.5-VL-7B · MathVista | accuracy: +1.1% | Helped |
| Add online multimodal transformations (ablation) | Share-GRPO on Qwen2.5-VL-7B · MathVista | accuracy: +0.9% | Helped |
| Add local advantage estimation to global advantage estimation (ablation) | Share-GRPO on Qwen2.5-VL-7B · MathVista | accuracy: 74.8 -> 75.4 | Helped |
| Increase question variants from 2 to 3 (ablation) | Share-GRPO · MathVista | accuracy: 75.4 -> 75.6 | Helped |
| Increase question variants from 3 to 4 (ablation) | Share-GRPO · MathVista | accuracy: 75.6 -> 75.9 | Helped |
| Increase GRPO sampling from 6 to 12 (ablation) | GRPO · MathVista | accuracy: 72.3 -> 72.8 | Helped |
| Increase GRPO sampling from 12 to 24 (ablation) | GRPO · MathVista | accuracy: 72.8 -> 73.0; 0.2% | Helped |
Surprising findings
- Plain GRPO sampling scaled to 24 responses gives only marginal gains (72.3 to 73.0 on MathVista), suggesting scaling samples has a ceiling.
- Share-GRPO is complementary to dynamic sampling, with the combination reaching 67.2 average versus 65.1 for GRPO with dynamic sampling alone.
Limitations
- Hierarchical advantage uses responses from multiple variants, roughly doubling rollouts per question relative to GRPO with the same n, increasing compute.
- Online visual transformations (rotation, noise) are manually selected and require manual textual prompts per transformation, limiting generality.
- Evaluation is limited to a single RL dataset source (MM-Eureka 52K) and two model sizes.
Experimental setup
Datasets: MM-Eureka training data (52K sampled)
Benchmarks: MathVista MMStar MMMU MathVerse MathVision AI2D
Metrics: accuracy average accuracy
Baselines: GPT-4o Claude3.7-Sonnet Kimi1.5 LLaVA-Reasoner-8B LLaVA-CoT-11B Mulberry-7B X-REASONER-7B R1-Onevision-7B Vision-R1-7B OpenVLThinker-7B MM-Eureka-7B ThinkLite-7B
Builds on: GRPO PPO DeepSeek-R1 EasyR1
Closest papers by meaning
- Revisual-R1: Advancing Multimodal Reasoning From Optimized Cold Start to Staged Reinforcement Learning (ICLR 2026 · similarity 0.90)
- SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Stable and Efficient Single-Rollout RL for Multimodal Reasoning (CVPR 2026 · similarity 0.90)
- Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (ICLR 2026 · similarity 0.90)
- SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.