Research map / Vision-language and multimodal / Multimodal LLMs

AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AlloSpatial, an agentic framework with World2Mind (a cognitive-mapping tool producing Allocentric-Spatial Trees and route maps) and a three-stage Spatial Reasoning Harness, improving proprietary models training-free and training Qwen3-VL agents via SFT cold start plus GSPO with a harness-gated trajectory reward.

Problem

Multimodal foundation models struggle to convert local egocentric observations into global allocentric spatial representations, limiting spatial reasoning on VSI-Bench and MindCube.

Approach

Provide a query-conditioned allocentric cognitive mapping tool (AST plus route map) that models call inside a judge-collect-arbitrate harness, and internalize this into open-weight models via cold-start SFT and GSPO with a harness-gated trajectory reward that credits accuracy only when the reasoning structure is followed.

Key results

• Training-free World2Mind+harness improves Claude-4.6-Opus on VSI-Bench overall from 38.4 to 56.0 (+17.7) and GPT-5.2 by +7.3, Gemini-3-Pro by +5.8.
• AlloSpatial-8B reaches 54.2 overall on VSI-Bench (7 frames), above Qwen3-VL-32B (53.1), Spatial-MLLM-4B (49.1), and Cambrian-S-3B (49.5).
• AlloSpatial-4B reaches 69.1% overall on MindCube, versus 28.3 for base Qwen3-VL-4B; QA-only SFT reaches 53.9 on MindCube but drops VSI-Bench to 43.1.

Surprising findings

• Text-only AST inputs (blind setting) substantially improve spatial reasoning over text-only baselines, suggesting allocentric structure alone carries much of the signal.
• QA-only SFT improves MindCube strongly but decreases VSI-Bench from 45.1 to 43.1, indicating answer supervision alone does not generalize.

Limitations

• Weak on fine-grained numerical tasks (absolute distance, object counting) due to reconstruction drift and imperfect metric calibration.
• Online World2Mind execution adds substantial inference and training cost (multi-process service on NPUs).
• Evaluation on only two benchmarks with tiny splits.

Experimental setup

Datasets: VSI-590K MindCube training split VSI-Bench MindCube

Benchmarks: VSI-Bench MindCube

Metrics: pass@1 accuracy mean relative accuracy (MRA) overall unweighted average

Baselines: GPT-5.2 Claude-4.6-Opus Gemini-3-Pro Gemini-2.5-Pro Qwen3-VL-32B Spatial-MLLM Cambrian-S Think3D

Builds on: GSPO ReAct Think3D SAM 3

Closest papers by meaning

• Active Exploring like a Pigeon: Reinforcing Spatial Reasoning via Agentic Vision-Language Models (ICML 2026 · similarity 0.89)
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning (NeurIPS 2026 · similarity 0.89)
• Allocentric Perceiver: Disentangling Allocentric Reasoning from Egocentric Visual Priors via Frame Instantiation (NeurIPS 2026 · similarity 0.89)
• MindJourney: Test-Time Scaling with World Models for Spatial Reasoning (NeurIPS 2025 · similarity 0.88)
• Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.88)
• From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs (CVPR 2026 · similarity 0.88)