Research map / Vision-language and multimodal / Multimodal LLMs
From Views to Worlds: Active Exploration over 3D Worlds for Vision-Language Models
Qijian Tian, Jiayu Ying, Ke Fan, Lizhuang Ma, Xin Tan
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes active exploration over 3D worlds so vision-language models can acquire spatial evidence beyond currently visible views for 3D spatial reasoning.
Problem
Passive view-centric perception limits VLMs' 3D spatial reasoning, which often requires actively acquiring evidence beyond visible views (camera poses, cross-view relations).
Approach
Shifts VLMs from passive view-centric perception to active exploration over 3D worlds, letting the model acquire additional spatial evidence such as camera poses and cross-view relationships to infer 3D relations not explicit in the input views.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual and linguistic world interfaces with active exploration (training-free) | Qwen3-VL-8B · MindCube-Tiny | overall accuracy: 30.6 -> 50.2 | Improved |
| use V2W active exploration instead of chain-of-thought prompting | Qwen3-VL-235B · MindCube-Tiny | overall accuracy: 45.4 -> 59.6 | Improved |
| add chain-of-thought prompting | Qwen2.5-VL-7B · MindCube-Tiny | overall accuracy: 34.5 -> 29.1 | Hurt |
| add both interfaces in training-free V2W (ablation) | Qwen2.5-VL-7B · MindCube-Tiny Rotation | Rotation accuracy: 35.0 -> 35.0 | No clear effect |
| add the linguistic interface to the visual interface (ablation) | V2W training-free · MindCube-Tiny | overall accuracy: 38.5 -> 43.8 | Improved |
| add the full world interface after SFT-then-RL (ablation) | V2W · MindCube-Tiny | overall accuracy: 74.8 -> 90.8 | Improved |
| use RL instead of SFT for exploration policy (ablation) | V2W with full interface · MindCube-Tiny | overall accuracy: 71.3 -> 83.9 | Improved |
| add RL after SFT (ablation) | V2W with full interface · MindCube-Tiny | overall accuracy: 71.3 -> 90.8 | Improved |
| add step-wise reward in RL (ablation) | V2W SFT-then-RL · MindCube-Tiny | overall accuracy: 89.0 -> 90.8 | Improved |
| swap Depth Anything 3 for VGGT in the 3D world construction (ablation) | V2W training-free · MindCube-Tiny | overall accuracy: 43.8 -> 42.6 (Qwen2.5-VL-7B); 50.2 -> 50.0 (Qwen3-VL-8B) | No clear effect |
| use V2W with larger Qwen3-VL series vs Qwen2.5-VL-7B | V2W training-free · MindCube-Tiny | overall accuracy gain: +9.3 (Qwen2.5-VL-7B) vs +19.6 (Qwen3-VL-8B) | Improved |
| add V2W to GPT-4.1-mini over pySpatial | GPT-4.1-mini · MindCube-Tiny | overall accuracy: 58.2 -> 61.7 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- pySpatial — pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoning (ICLR 2026) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- MindJourney: Test-Time Scaling with World Models for Spatial Reasoning (NeurIPS 2025 · similarity 0.92)
- Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators (NeurIPS 2026 · similarity 0.92)
- From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.91)
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.91)
- Abstract 3D Perception for Spatial Intelligence in Vision-Language Models (CVPR 2026 · similarity 0.91)
- Active Exploring like a Pigeon: Reinforcing Spatial Reasoning via Agentic Vision-Language Models (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
