Research map / Vision-language and multimodal / Multimodal LLMs

Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Scene Graph Thinking (SaGe), which builds hierarchical scene graphs from image-text data, samples 120K node-articulated CoT samples for SFT, and applies GRPO with node-as-proxy relevance and grounding rewards to improve fine-grained and spatial visual reasoning in Qwen2.5-VL models.

Problem

Existing think-with-image MLLMs focus on isolated objects via cropping and zoom-in, neglecting structured relationships among scene elements, which leads to inefficient target navigation and weak fine-grained and holistic reasoning.

Approach

Pre-construct hierarchical scene graphs (nodes with attributes, bboxes, depth ranges; edges with spatial/interaction/semantic relations) with an automated pipeline, sample node-articulated chain-of-thought traces from them for cold-start SFT, and then optimize with GRPO using node-relevance and node-grounded rewards that treat nodes as proxies for valid navigation paths.

Key results

• SaGe-3B reaches 89.0 on V*Bench, up from 75.4 for Qwen2.5-VL-3B, and SaGe-7B reaches 89.0 vs 76.4 for Qwen2.5-VL-7B.
• SaGe-3B gets 77.8 on CVBench-2D (vs 67.0 for Qwen2.5-VL-3B) and 72.3 on CVBench-3D (vs 66.5).
• SaGe-7B gets 76.5 on HRBench-4K and 73.4 on HRBench-8K, improving over Qwen2.5-VL-7B by 7.7 and 8.1 points.

Surprising findings

• Node-grounded reward alone consistently degrades performance, which the authors attribute to reward hacking.
• SaGe-7B scores below its teacher Qwen2.5-VL-72B on CVBench-3D distance (72.2 vs 83.8).

Limitations

• Scene graph construction depends on a 72B teacher and proprietary-level pipeline, making it costly to reproduce.
• Depth cues are coarse ranges rather than precise estimates; node recall is not measured.
• SaGe-7B underperforms the teacher on CVBench-3D distance.

Experimental setup

Datasets: SA-1B COCO 2014 V* train set grounding data chart data

Benchmarks: V*Bench HRBench-4K HRBench-8K CVBench-2D CVBench-3D MMStar RefCOCO ChartQA

Metrics: accuracy Acc@0.5

Baselines: GPT-4o Gemini-2.5-Pro GPT-5-Mini Qwen2.5-VL InternVL3 LLaVA-OneVision DeepEyes Pixel-Reasoner LVR Vision-R1 Thyme

Builds on: GRPO Qwen2.5-VL Depth Anything SAM

Closest papers by meaning

• Compositional Chain-of-Thought Prompting for Large Multimodal Models (CVPR 2024 · similarity 0.91)
• Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models (ICML 2025 · similarity 0.91)
• Mario: Multimodal Graph Reasoning with Large Language Models (CVPR 2026 · similarity 0.91)
• Synthetic Visual Genome (CVPR 2025 · similarity 0.90)
• GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning (CVPR 2026 · similarity 0.90)
• DiGraphHal-Bench: Evaluating Multimodal Large Language Models on Complex Directed Graphs (CVPR 2026 · similarity 0.90)