Research map / Vision-language and multimodal / Multimodal LLMs

AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains Qwen2.5-VL models to plan multi-turn visual tool use via cold-start SFT on synthesized trajectories, Tool-GRPO RL with hierarchical tool rewards, and randomized-identifier adaptive learning for generalization to unseen tools and tasks.

Problem

Existing tool-augmented MLLMs rely on single atomic tools or rigid invocation patterns and generalize poorly to multi-turn tool planning, unseen tools, and new tasks.

Approach

Combines a multi-turn tool trajectory curation pipeline (with reflection and tool-failure cases), a GRPO variant (Tool-GRPO) with format-gated, hierarchical tool-call rewards and asymmetric adaptive tool reward, and adaptive learning that randomizes tool/argument names and paraphrases descriptions so the policy relies on semantics rather than identifiers.

Key results

• AdaReasoner 7B reaches 97.64 on VSP and 96.60 on Jigsaw in single-task setting (TC+TG), above GPT-5's 80.10 on Jigsaw-related comparison; the main table reports 88.60 on Jigsaw for the generalized model.
• Generalized AdaReasoner 7B averages 76.49 across eight benchmarks vs 51.80 for Qwen2.5-VL 7B (+24.72), and beats GPT-5 + Tools (74.83) on average.
• Randomized TC + Randomized TG lifts unseen-task VSP from 28.09 to 78.91 when trained only with Jigsaw cold-start data.

Surprising findings

• Inference-time use of an unseen A* tool degraded verification performance (94.20 to 80.00) without RL, showing zero-shot tool adaptation is unstable.
• Standard (non-randomized) TC+TG generalized poorly to unseen tasks, sometimes degrading performance below base.

Limitations

• Single-task and generalization results rely on a small set of hand-designed tools and synthetic environments (FrozenLake/VSP, Jigsaw-COCO).
• Generalization to unseen tools is shown only with equivalent-functionality renamed tools, not truly novel capabilities.
• Some tasks (e.g., WebMMU, GUIChat) show smaller gains or lose to proprietary models.

Experimental setup

Datasets: VSP (FrozenLake procedurally generated via Gymnasium) Jigsaw-COCO (from COCO-2017) GUIChat WebMMU Visual Search data

Benchmarks: VSPO VSP Jigsaw-COCO BLINK-J GUIChat WebMMU HRBench V*

Metrics: accuracy average score calls per sample (CPS) tool success rate navigation and verification scores

Baselines: Direct SFT Direct GRPO GPT-5 Claude Sonnet 4 Gemini 2.5 Flash DeepEyes PixelReasoner Qwen2.5-VL 72B

Builds on: GRPO DeepSeek-R1 DeepEyes Pixel-Reasoner

Closest papers by meaning

• PixelCraft: A Multi-Agent system for High-Fidelity Visual Reasoning on Structured Images (ICLR 2026 · similarity 0.91)
• Visual Planning: Let's Think Only with Images (ICLR 2026 · similarity 0.90)
• MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods (NeurIPS 2026 · similarity 0.90)
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.90)
• RegionReasoner: Region-Grounded Multi-Round Visual Reasoning (ICLR 2026 · similarity 0.90)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.90)