Research map / Vision-language and multimodal / Multimodal LLMs

Learning to Select Visual Tools from Experience

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains a Qwen2.5-VL-7B tool-selection agent with GRPO-style RL from task-outcome rewards to choose from a 23-tool visual library, while keeping the reasoner frozen and transferable across reasoners.

Problem

Existing tool-augmented visual reasoning relies on training-free prompting or supervised fine-tuning with human demonstrations, which lacks active exploration and adaptive selection among many tools with varying quality.

Approach

Treat tool selection as an RL policy optimized by verifiable outcome rewards (whether tools flip the reasoner's answer from wrong to right, or hurt it), with a frozen reasoner so the learned selector transfers to other VLMs; optionally multi-turn refinement gated by the reasoner's self-reported confidence.

Key results

• Multi-turn VisTA with QwenVL-7B agent and reasoner reaches 79.9 on ChartQA, 75.8 on ChartQA-OoD, 57.0 on Geometry3K, and 52.1 on MathVerse, beating training-free and GRPO-trained baselines.
• Transferring the QwenVL-trained selector to a frozen GPT-4o reasoner yields 88.1 ChartQA, 75.6 ChartQA-OoD, and 52.0 Geometry3K, beating the best GPT-4o training-free baseline.
• On ChartQA-OoD the multi-turn agent improves over the GRPO-trained reasoner by 11.5 points (75.8 vs 64.3).

Surprising findings

• Tool-selection policies trained with a QwenVL reasoner transfer to GPT-4o without retraining and still improve results.
• Training-free GPT-4o tool selection performed worse than the QwenVL-7B agent, and its selections showed no clear alignment with tool effectiveness.

Limitations

• Tools are abstracted as black-box modules chosen by index; no parameterized argument generation (e.g., bounding boxes for zoom).
• Reported results for the 3-turn setting and the ChartQA-OoD gains rely on a single run with no variance reported.
• Tool pool is a fixed set of 23 tools drawn from specific pretrained models; generalization beyond these benchmarks is only partially tested.

Experimental setup

Datasets: ChartQA Geometry3K BlindTest

Benchmarks: ChartQA ChartQA-OoD Geometry3K MathVerse BlindTest

Metrics: accuracy

Baselines: QwenVL 7B training-free GPT-4o tool selection GRPO-trained reasoner GPT-4o Claude-3.5 Sonnet Molmo-72B InternVL2-76B

Builds on: GRPO DeepSeek-R1 ReTool

Closest papers by meaning

• Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs (CVPR 2026 · similarity 0.90)
• Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents (ICML 2026 · similarity 0.89)
• CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization (CVPR 2026 · similarity 0.89)
• SenseSearch: Empowering Vision-Language Models with High-Resolution Agentic Search-Reasoning via Reinforcement Learning (CVPR 2026 · similarity 0.88)
• GUI-Eyes: Tool-Augmented Perception for Visual Grounding in GUI Agents (AAAI 2026 · similarity 0.87)
• ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoning (CVPR 2026 · similarity 0.87)