Research map / Vision-language and multimodal / Multimodal LLMs

From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SFI-Bench, a video MCQ benchmark (1,555 questions from 134 egocentric indoor scans) that tests spatial cognition and functional/knowledge-grounded reasoning in MLLMs, and finds that current models struggle with counting, temporal continuity, and web-search-dependent functional tasks.

Problem

Existing video benchmarks mainly test geometric perception and factual recall, not higher-order spatial memory, affordance/functional reasoning, and integration with external knowledge needed for grounded agentic intelligence.

Approach

Reformulates counting, path, and layout tasks as compositional cognition-level problems and adds functional tasks (functional association, operation planning, troubleshooting) that require web search for manuals, evaluated together in one video MCQ benchmark.

Key results

• Gemini-3.1-Pro achieves the strongest overall performance among evaluated models, with global conditional counting the main bottleneck across all models.
• GPT-5 with web search significantly outperforms its offline version on operation planning and troubleshooting under high reasoning budgets, with a gap of up to 8%.
• GPT-5 with visual input reaches 75.3% on functional tasks versus 67.6% with caption-only input, and layout accuracy drops from 83.0% to 51.6%.

Surprising findings

• Shuffling frame order up to 100% causes little degradation in GPT-5 accuracy, suggesting models do not rely on temporal continuity.
• Longer reasoning chains do not improve accuracy; reasoning-enabled open-source models show minimal gains over instruct counterparts.

Limitations

• Benchmark is limited to 134 egocentric indoor videos from two datasets.
• Question generation relies on Gemini-2.5-Pro metadata extraction, which may introduce bias.
• Web-search tasks depend on live online manuals, making results time-sensitive and hard to reproduce.

Experimental setup

Datasets: ARKitScenes ScanNet++

Benchmarks: SFI-Bench

Metrics: accuracy macro-average accuracy

Baselines: Gemini-3.1-Pro Gemini-2.5-Pro Gemini-2.5-Flash GPT-5 GPT-5.4 o4-mini Qwen3-VL InternVL-3.5 GLM-4.5 LLaVA-OneVision LLaVA-Video

Builds on: VSI-Bench

Closest papers by meaning

• SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence (CVPR 2026 · similarity 0.91)
• Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.91)
• MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence (ICLR 2026 · similarity 0.91)
• Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests (NeurIPS 2026 · similarity 0.91)
• EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs (CVPR 2026 · similarity 0.91)
• GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning (ICML 2026 · similarity 0.91)