Research map / Vision-language and multimodal / Multimodal LLMs

SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SpatiaLQA, a 9,605-QA benchmark for spatial logical reasoning in indoor scenes, evaluates 41 VLMs with GPT-4o-based step matching, and proposes recursive scene graph assisted reasoning (RSGAR) that improves GPT-4o.

Problem

Vision-language models struggle with spatial logical reasoning, which requires combining spatial understanding of object relations with ordered, dependency-aware multi-step reasoning in real indoor scenes; existing benchmarks do not measure this jointly.

Approach

Build an open-vocabulary multi-step QA benchmark with step-level preconditions, score predictions via GPT-4o step matching plus Hungarian one-to-one matching, and improve VLMs by iteratively generating a task-relevant scene graph (from source to target objects) using depth and segmentation maps before answering.

Key results

• GPT-5 achieves the best F1 among evaluated VLMs at 76.0 content F1 and 47.0 precondition F1, with humans at 97.6 and 92.5.
• RSGAR with GPT-4o improves content F1 from 67.4 to 69.8 and precondition F1 from 25.1 to 28.1, outperforming PhysAgent and CoT baselines.
• Adding only depth or segmentation maps to GPT-4o lowers performance (e.g., seg-only content F1 drops to 50.3), indicating raw visual cues do not help.

Surprising findings

• Adding depth or segmentation maps directly to vanilla GPT-4o prompts degrades performance, while the scene-graph reasoning helps.
• Precondition prediction is much weaker than content prediction across all 41 VLMs, indicating deficient causal reasoning.

Limitations

• Relies on GPT-4o as both evaluator and matching judge, with alignment to humans validated only on 300 samples and eight models.
• RSGAR is roughly 6x slower than vanilla reasoning at T=5 and the gains are modest (about +2 F1).
• Benchmark is limited to indoor scenes and a single-image, manipulation-style action space.

Experimental setup

Datasets: SpatiaLQA

Benchmarks: SpatiaLQA

Metrics: content recall content precision content F1 precondition recall precondition precision precondition F1

Baselines: PhysAgent Chain of Thought GPT-4o with depth map GPT-4o with segmentation map GPT-4o with seg and depth

Builds on: Depth Anything V2 SAM PhysAgent chain-of-thought

Closest papers by meaning

• SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities (CVPR 2024 · similarity 0.94)
• Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.93)
• SpatiaLab: Can Vision–Language Models Perform Spatial Reasoning in the Wild? (ICLR 2026 · similarity 0.93)
• Enhancing Spatial Reasoning Through Visual and Textual Thinking (AAAI 2026 · similarity 0.92)
• SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning (NeurIPS 2025 · similarity 0.92)
• SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.92)