Research map / Vision-language and multimodal / Multimodal LLMs

VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains VLMs with outcome-only GRPO to interleave text reasoning with one Python-based visual editing tool call (highlight/mask/draw on table or chart regions), reporting gains on chart and table QA.

Problem

VLM reasoning remains text-driven: images are encoded once and reasoning is text-only, while prior test-time visual-sketchpad methods need strong closed models and lack training for visual intermediate steps.

Approach

Apply RL finetuning (GRPO with outcome-only LLM-judge reward) to Qwen2.5-VL so the model learns, within rollouts, to call Python visual editing tools and reason over both the original and edited images, avoiding process-based reward hacking.

Key results

• VTool-R1 7B reaches 80.7% on the chart split and 71.7% on the table split, versus 76.2% and 64.7% for pure-run Qwen2.5-VL-7B.
• VTool-R1 3B improves from 51.8% to 64.0% on the chart split and from 41.3% to 57.9% on the table split, while prompted tool use alone drops accuracy to 24.6% and 24.3%.
• VTool-R1 32B (86.7% chart, 84.5% table) does not clearly beat pure-run Qwen2.5-VL-32B (88.0% chart), so gains are mostly at smaller scales.

Surprising findings

• Pure-run Qwen2.5-VL 3B and 7B outperform GPT-4o-based tool use and often GPT-4o pure run on these splits, suggesting possible VQA-style post-training.
• Process-based rewards (penalizing failed tool calls or rewarding successful calls) led to tool use collapsing to zero or reward hacking.

Limitations

• Only single-turn tool use with one tool call allowed; multi-turn editing left for future work.
• Tool-call success measured by a proxy (no exception and valid image returned), with no oracle verifier.
• Evaluated only on structured chart and table reasoning with a fixed set of simple highlight/mask/draw tools.

Experimental setup

Datasets: VWTQ VWTQ_syn VTabFact ChartQA Refocus data splits

Benchmarks: ChartQA (Refocus chart split) VWTQ / VWTQ_syn / VTabFact (Refocus table split)

Metrics: accuracy

Baselines: Qwen2.5-VL pure run Qwen2.5-VL prompted tool use R1-VL GPT-4o Deepeyes

Builds on: GRPO Refocus Visual Sketchpad Search-R1 DeepSeek-R1

Closest papers by meaning

• Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models (NeurIPS 2025 · similarity 0.91)
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.91)
• Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning (ICLR 2026 · similarity 0.91)
• Activating Visual Context and Commonsense Reasoning Through Masked Prediction in VLMs (AAAI 2026 · similarity 0.91)
• See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection (CVPR 2026 · similarity 0.91)
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning (NeurIPS 2024 · similarity 0.91)