Research map / Vision-language and multimodal / Multimodal LLMs
SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ThinkLite-VL, which uses MCTS iteration counts to select ~11k (7B) and 7.5k (72B) hard samples from 70k open-source visual reasoning examples and applies GRPO-based RFT to Qwen2.5-VL without distillation, reaching SoTA on MathVista.
Problem
Reinforcement fine-tuning of VLMs for visual reasoning is less effective than for LLMs, and existing pipelines rely on costly distillation and SFT; the question is how to select training samples efficiently for self-improvement.
Approach
Repurposes Monte Carlo Tree Search, self-rewarded with no PRM, to measure each sample's difficulty as the number of MCTS iterations the VLM needs to reach a correct answer, then keeps samples needing more than 5 iterations plus unsolved ones for GRPO-based RFT.
Key results
- ThinkLite-VL-7B reaches 75.1 on MathVista and improves the Qwen2.5-VL-7B-Instruct average from 59.69 to 64.18 across eight benchmarks, versus 60.89 for random 11k selection.
- ThinkLite-VL-72B reaches 79.7 on MathVista and 64.3 on MathVerse, with average 72.67 vs 68.25 for the base model.
- Training on MCTS-selected samples outperforms self-consistency-selected offline (63.15) and online (62.34) variants and the full 70k set (63.13) at 7B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select training samples with MCTS-guided difficulty filtering rather than random sampling | ThinkLite-VL-7B · 8 visual benchmarks average | Avg.: 60.89 -> 64.18 | Helped |
| Select training samples with MCTS-guided difficulty filtering rather than random sampling | ThinkLite-VL-72B · 8 visual benchmarks average | Avg.: 69.91 -> 72.67 | Helped |
| Combine unsolved samples with samples requiring more than five iterations (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg. score: 62.04 -> 63.89 | Helped |
| Expand the selected difficulty set from Iter5+Unsolved to the full 70k dataset (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg. score: 63.89 -> 63.13 | Hurt |
| Raise the iteration threshold from 5 to 10 while retaining unsolved examples (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg. score: 63.89 -> 62.65 | Hurt |
| Raise the iteration threshold from 5 to 20 while retaining unsolved examples (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg. score: 63.89 -> 62.61 | Hurt |
| Raise the iteration threshold from 5 to 30 while retaining unsolved examples (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg. score: 63.89 -> 62.39 | Hurt |
| Raise the iteration threshold from 5 to 40 while retaining unsolved examples (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg. score: 63.89 -> 62.26 | Hurt |
| Use only MCTS-unsolved examples instead of combining them with medium-difficulty examples (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg. score: 62.04 -> 63.89 | Hurt |
| Use self-consistency filtering instead of MCTS filtering (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg.: 63.15 -> 64.18 | Hurt |
| Train on the full 70k dataset instead of the MCTS-selected subset (ablation) | ThinkLite-VL-7B RFT · 8 visual benchmarks average | Avg.: 63.13 vs 64.18 | Hurt |
| Use the 72B model's selected data to train the 7B model (ablation) | ThinkLite-VL-7B · 8 visual benchmarks average | Avg.: 61.24 vs 64.18 | Hurt |
Surprising findings
- Training on the full 70k set or random subsets of easy samples raised training reward more but did not improve benchmark performance, suggesting reward is a poor proxy for reasoning gain.
- Sample sets selected by the 7B and 72B models transfer poorly to the other model, with cross-trained gains markedly smaller than self-selected gains.
Limitations
- Selection relies on Qwen2.5-7B-Instruct as critic for correctness, and the MCTS prompts use open-ended answer conversion that changes the original benchmark formats.
- Difficulty is defined relative to a specific policy model, so curated sets do not transfer well across model sizes.
- Comparisons with some baselines (e.g., Mulberry, Vision-R1) rely on reported numbers and different training data sizes, and gaps on some benchmarks are marginal.
Experimental setup
Datasets: Geometry3K GeoQA Geos FigureQA ScienceQA OK-VQA IconQA TabMWP
Benchmarks: MathVista MathVision MathVerse MMMU MMStar MMBench MM-Vet AI2D
Metrics: accuracy average accuracy
Baselines: Qwen2.5-VL-7B-Instruct Qwen2.5-VL-72B-Instruct ThinkLite-VL-Random11k ThinkLite-VL-Random7.5k LLaVA-Cot-11B Mulberry-7B Vision-R1-7B OpenVLThinker-7B MM-EUREKA-Qwen-7B QvQ-72B
Builds on: GRPO Monte Carlo Tree Search Easy-R1 Qwen2.5-VL
Closest papers by meaning
- Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models (NeurIPS 2025 · similarity 0.91)
- Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training (ICML 2026 · similarity 0.91)
- MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods (NeurIPS 2026 · similarity 0.90)
- Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale (ICML 2026 · similarity 0.90)
- AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.90)
- VGR: Visual Grounded Reasoning (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.