Research map / Vision-language and multimodal / Multimodal LLMs

Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Transductive Visual Programming, which solves 3D spatial questions with basic tools, stores successful programs in an Example Library, and abstracts recurring patterns into validated reusable tools in a Tool Library.

Problem

Visual programming for 3D spatial reasoning is limited by fixed toolsets or speculative (inductive) tool creation before solving problems, leading to suboptimal programs and poor tool reuse.

Approach

Tools are created transductively: abstracted from clusters of the system's own high-quality solved programs (experience), validated by execution and correctness checks, and merged over time, forming a closed program-tool-program loop.

Key results

• TVP reaches 33.3% overall accuracy on Omni3D-Bench, vs. 29.9% for VADAR and 27.2% for GPT-4o.
• Abstracted tools are used as core program dependency about 5x more often than inductively created ones; 36.3% of programs use only abstracted tool calls.
• On SpatialScore-Hard zero-shot transfer, TVP achieves 52.3% overall accuracy vs. 32.8% for VADAR.

Surprising findings

• Random datapoint ordering ultimately produced more tools (61) and better long-run performance than curriculum ordering (51 tools).
• The Example Library alone (without tool creation) already outperforms all prior baselines on Omni3D-Bench (31.7%).

Limitations

• Building libraries costs about $80 per iteration and roughly 7 hours of sequential runtime.
• Tool creation depends on proprietary GPT-4o and o4-mini for generation, judging and abstraction; open-source 32B backbone only approaches GPT-4o performance.
• Absolute accuracy remains low (33.3% on Omni3D-Bench), so precise spatial reasoning is still far from solved.

Experimental setup

Datasets: Omni3D-Bench 3DSR-Bench SpatialSense VG-Bench SpatialScore-Hard

Benchmarks: Omni3D-Bench SpatialScore-Hard

Metrics: accuracy Mean Relative Accuracy Float (±10%) accuracy cyclomatic complexity

Baselines: GPT-4o LLaVA-OneVision-7B-Chat Qwen2-VL-7B-Instruct Molmo-7B-D SpaceMantis SpatialBot-3B VisProg ViperGPT VADAR

Builds on: VADAR ViperGPT VisProg LILO

Closest papers by meaning

• AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning (ICLR 2026 · similarity 0.89)
• CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoning (CVPR 2026 · similarity 0.88)
• Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning (NeurIPS 2026 · similarity 0.88)
• Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.88)
• SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning (NeurIPS 2025 · similarity 0.87)
• SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.87)