Research map / Large language models / Reasoning and chain of thought

Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AutoTool, an RL-trained (GRPO) Qwen2.5-VL-7B that explicitly chooses between tool-on (zoom-in) and tool-off text-only reasoning per query, using mode-specific rewards and an adaptive mode-balancing coefficient.

Problem

Existing tool-augmented MLLM reasoning models (e.g., DeepEyes) always invoke tools regardless of need, increasing training/inference cost and sometimes introducing distracting visual evidence and hallucinations.

Approach

Explicit <tool_on>/<tool_off> control tokens define two reasoning modes; mode-specific reward (penalize wrong answers after tool use, reward correct tool-off answers) plus Adaptive Mode Balancing that adjusts tool reward coefficient to keep mode frequencies balanced early, then freed in the final stage.

Key results

• AutoTool-7B reaches 90.1% overall on V* versus 69.1% for the Qwen2.5-VL-7B base (about +21 points) and 87.4% for DeepEyes.
• Training time drops from 44.9 h (DeepEyes) to 35.8 h (AutoTool), a 20.3% reduction; POPE random-split inference time falls 44.9%.
• AutoTool-7B achieves 74.0% overall on HRBench-8K and 76.9% on HRBench-4K, both above DeepEyes (71.5 and 74.9).

Surprising findings

• Removing the mode-balancing constraint from step 0 leads to premature collapse toward tool-off mode and weaker fine-grained perception.
• Setting the base tool coefficient to 0 causes reward hacking where the model claims tool-on mode but makes no valid tool calls.

Limitations

• Tool set limited to a single zoom-in operation.
• Reliance on a 72B LLM judge for accuracy reward adds cost and noise sensitivity.
• Reward and balancing hyperparameters (lambda_base, penalty) require tuning; extreme values cause collapse.

Experimental setup

Datasets: V* ArxivQA ThinkLite-VL

Benchmarks: V* HRBench-4K HRBench-8K RefCOCO RefCOCO+ RefCOCOg ReasonSeg POPE MathVista MathVerse MathVision WeMath

Metrics: accuracy IoU@0.5 training time inference time

Baselines: DeepEyes Qwen2.5-VL-7B Qwen2.5-VL-32B InternVL3-8B InternVL3-38B ZoomEye Pixel-Reasoner SEAL DyFo GPT-4o LLaVA-OneVision LlamaV-o1

Builds on: GRPO DeepEyes DeepSeek-R1

Closest papers by meaning

• AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning (ICML 2026 · similarity 0.91)
• Joint Optimization of Tool Creation and Use for Large Language Model Agents (NeurIPS 2026 · similarity 0.90)
• Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning (ICLR 2026 · similarity 0.90)
• ReTool: Reinforcement Learning for Strategic Tool Use in LLMs (ICLR 2026 · similarity 0.89)
• ToolRL: Reward is All Tool Learning Needs (NeurIPS 2025 · similarity 0.89)
• Eliciting Reasoning in Language Models with Cognitive Tools (NeurIPS 2025 · similarity 0.89)