Research map / Vision-language and multimodal / Multimodal LLMs

Allocentric Perceiver: Disentangling Allocentric Reasoning from Egocentric Visual Priors via Frame Instantiation

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Allocentric Perceiver, a training-free pipeline that lifts images to a metric 3D world via off-the-shelf geometry experts, instantiates a query-conditioned object-centric frame, and prompts a VLM with text-only geometric descriptions to improve allocentric spatial reasoning.

Problem

VLMs perform poorly on allocentric spatial queries requiring perspective shifts, as egocentric visual priors from image-text training data interfere with target-centric reasoning.

Approach

Recover metric 3D object states with depth/pose experts, transform coordinates into a target-object-centric frame (origin at reference object, forward axis from orientation or inter-object vector), and feed the VLM only structured geometric text so it performs explicit computation rather than implicit mental rotation; a query router splits allocentric and egocentric streams.

Key results

• Alloceiver improves allocentric average accuracy by +8.33 (Qwen2.5-VL-7B), +10.84 (Qwen2.5-VL-32B), +10.73 (InternVL2.5-38B), and +10.98 (GPT-4o) points.
• Qwen2.5-VL-7B + Alloceiver (41.25% allocentric avg) outperforms unaugmented GPT-4o (39.81%) on allocentric tasks.
• Egocentric accuracy also rises by 2.21 to 8.28 points across backbones.

Surprising findings

• Removing visual input from Qwen2.5-VL-7B lowered egocentric accuracy by 14.80% but slightly raised allocentric accuracy by 1.12%.
• Text-only geometric prompting performs about as well as adding the image back (average differs by roughly 1.5%).

Limitations

• Relies on the accuracy of off-the-shelf segmentation, depth, and orientation experts; errors propagate into geometric prompts.
• Evaluated only on multiple-choice subsets of two benchmarks.

Experimental setup

Datasets: ViewSpatial-Bench 3DSRBench

Benchmarks: ViewSpatial-Bench 3DSRBench

Metrics: accuracy

Baselines: GPT-4o GPT-5-mini Gemini-2.5-Pro InternVL2.5-38B Spatial-SSRL-7B SenseNova-SI-1.1-7B SpatialThinker-7B RoboBrain2.0-32B Cosmos-Reason1-7B REVPT-7B

Builds on: Depth-Anything-3 LangSAM CLIP DBSCAN

Closest papers by meaning

• Keep it SymPL: Symbolic Projective Layout for Allocentric Spatial Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.92)
• 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.89)
• AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Models (NeurIPS 2026 · similarity 0.89)
• SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs (ICLR 2026 · similarity 0.88)
• Pursuing Minimal Sufficiency in Spatial Reasoning (ICLR 2026 · similarity 0.88)
• Mind the GAP: Glimpse-based Active Perception improves generalization and sample efficiency of visual reasoning (ICLR 2025 · similarity 0.88)