Research map / Vision-language and multimodal / Multimodal LLMs

SIFThinker: Spatially-Aware Image Focus for Visual Reasoning

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SIFThinker, a spatially-aware think-with-images MLLM that interleaves depth-enhanced bounding boxes with text, trained via SFT on a new SIF-50K CoT dataset then GRPO-SIF with grounding, depth, and progressive-answer rewards.

Problem

MLLMs struggle with complex visual tasks such as spatial understanding and fine-grained perception, and existing think-with-images methods lack attention correction guided by spatial (depth) cues and often separate grounding from answer generation.

Approach

Interleave depth-annotated bounding boxes with natural language in the reasoning chain, generate such chains by reverse box expansion plus forward inference, and use RL rewards that jointly score format, hierarchical box IoU with a correction term (end vs. initial box), depth consistency, and a progressive answer score judged by an external VLM.

Key results

• On SAT-Static, SIFThinker-Qwen2.5-VL-7B reaches 72.8 vs. 65.5 for the Qwen2.5-VL-7B base and 67.4 for o3.
• On V*Bench Attribute, SIFThinker-7B scores 0.791 vs. 0.748 for SEAL and 0.739 for o3.
• On OVDEval, SIFThinker reaches NMS-AP 37.8 vs. 25.9 for VisRL trained on the same data.

Surprising findings

• Removing the depth input (w/o D_I) lowers VisCoT_s average from 0.760 to 0.655, yet removing the depth-consistency reward only drops it to 0.753, with some subsets slightly improving.
• SFT alone with CoT data (0.582) gives gains over VQA-SFT (0.536) but degrades on some subsets such as GQA, with RL providing the bulk of the improvement.

Limitations

• Trained only on single-image data, so may not generalize to multi-image dynamic spatial reasoning.
• Depends on an external proprietary judge (Doubao-1.5-vision-pro) for answer rewards and data generation.
• Some reported comparisons are under differing base models and training data, and SpatialBench/SAT numbers are inconsistent across tables (e.g., 64.3 vs 74.5 in text).

Experimental setup

Datasets: SIF-50K VisCoT TallyQA Flickr30k Visual7W GQA Open Images VSR Birds-200-2011 RefCOCO RefCOCO+ RefCOCOg

Benchmarks: SpatialBench SAT CV-Bench VisCoT_s V*Bench RefCOCO RefCOCO+ RefCOCOg OVDEval MME MMBench SEED-Bench

Metrics: accuracy Accuracy@0.5 NMS-AP F1

Baselines: o3-2025-04-16 VisCoT VisRL SEAL VLM-R1 SpatialBot SSR UNINEXT Grounding-DINO Qwen2.5-VL-7B

Builds on: GRPO VisCoT VisRL VLM-R1 SpatialBot DepthAnythingV2

Closest papers by meaning

• Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas (ICML 2025 · similarity 0.92)
• VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.91)
• Visual Structures Help Visual Reasoning: Addressing the Binding Problem in LVLMs (NeurIPS 2025 · similarity 0.91)
• SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning (NeurIPS 2025 · similarity 0.90)
• From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models (ICML 2026 · similarity 0.90)
• Perception Tokens Enhance Visual Reasoning in Multimodal Language Models (CVPR 2025 · similarity 0.90)