Research map / Vision-language and multimodal / Multimodal LLMs

SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DIRL, a two-phase (SFT teaching then interactive multi-tool RL exploration) training framework plus Toolshed tool-hosting platform, yielding SpaceTools-3B, a tool-using VLM for spatial reasoning and robot manipulation.

Problem

VLMs lack metrically precise spatial reasoning, and RL with many tools has a combinatorially large search space in which exploration fails, so multi-tool use must be learned without fixed pipelines or handcrafted prompting.

Approach

A two-phase double interactive RL scheme: first SFT on mixed traces from a single-pointing-tool IRL teacher and a frontier universal teacher using all tools, then resume interactive GRPO with the full toolset; tools are served live via the Toolshed platform during training.

Key results

• SpaceTools-3B reaches 70.00% overall on RoboSpatial-Home, beating Gemini-ER 1.5 (62.50%) and tool-free SFT/RL by about 12 and 16 points.
• SpaceTools-3B achieves 34.37% pose IoU on BOP-ASK versus 9.03% for GPT-5 and 1.67% for Claude Sonnet 4.5.
• In real-robot tests SpaceTools reaches 86% pick, 83% relational pick and 86% pick-and-place success, matching or beating Claude Sonnet 4.5 + Toolshed on pick and pick-and-place.

Surprising findings

• Adding Toolshed tools to GPT-5 and Claude degrades RoboSpatial performance (58.39 to 55.14 and 57.43 to 52.86), attributed to tool overuse and misreading tool outputs.
• A model trained only with RoboSpatial pointing IRL transfers to RefSpatial (34.3%) where other fine-tuning approaches score zero.

Limitations

• Tools return mostly structured text or variables; reasoning over image outputs from tools is not explored.
• Real-robot tasks are short-horizon; mock robot tools were used during training rather than real robot execution.
• Tool overuse and difficulty interpreting tool outputs hurt some tasks for proprietary models with tools.

Experimental setup

Datasets: RoboSpatial RefSpatial BOP-ASK HOPE

Benchmarks: RoboSpatial-Home BLINK RefSpatial CVBench BOP-ASK

Metrics: normalized accuracy IoU MACE success rate

Baselines: Claude Sonnet 4.5 GPT-4o GPT-5 Gemini-ER 1.5 LLaVA-NeXT-8B Qwen2.5-VL-32B Qwen2.5-VL-3B SpaceLLaVA-13B RoboPoint-13B Molmo-7B RoboBrain2.0-7B RoboRefer-8B-SFT

Builds on: GRPO ViGoRL RoboRefer VERL LLaMA-Factory

Closest papers by meaning

• Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes (ICLR 2026 · similarity 0.90)
• SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.89)
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning (NeurIPS 2026 · similarity 0.89)
• RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics (CVPR 2025 · similarity 0.89)
• MindJourney: Test-Time Scaling with World Models for Spatial Reasoning (NeurIPS 2025 · similarity 0.89)
• Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing (NeurIPS 2025 · similarity 0.89)