AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Seeing Without Understanding: Disentangling Perception, Reasoning, and Simulation in VLM Gameplay

Dingyang Jin, Jiawei He, Calvin Lo, Steven Hu, Ryan Rad

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a two-stage diagnostic framework that separates VLM game-playing failures into perception, rule comprehension, and reasoning components.

Problem

VLMs underperform on game-based reasoning, but existing evaluations conflate failures in visual perception, rule comprehension, and reasoning.

Approach

A two-stage diagnostic: controlled perception tests isolate visual encoding, and a diagnostic matrix with a six-level rule-comprehension scale separates rule understanding from reasoning and simulation.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
align pieces with patch boundaries vs centers (ablation)VLMs (6 models) · Chess and Gomoku perceptionpiece accuracy: within 3-5 pointsNo clear effect
use non-divisible instead of divisible image resolution (ablation)VLMs (6 models) · Chess and Gomoku perceptionpiece accuracy: within 1-2 pointsNo clear effect
lower image resolution (ablation)GPT-5.2 · Gomoku perceptionpiece accuracy: 75% -> 32%Hurt
use 3D realistic rendering instead of 2D (ablation)VLMs (6 models) · Chess and Gomoku perceptionpiece accuracy: 1-5 points, inconsistentNo clear effect
increase piece density (ablation)VLMs (6 models) · Gomoku perceptionpiece accuracy: e.g. GPT-5.2 54.83 -> 79.75Improved
scale Qwen3-VL from 30B to 235B (ablation)Qwen3-VL · Rule complexity ladder Explicit modeaccuracyNo clear effect
increase rule complexity from L1 to L4 (ablation)Qwen3-VL 235B · Rule complexity ladder Predictivereasoning accuracy given verified perception: 99% -> 56%Hurt
use predictive instead of explicit mode (simulation) (ablation)VLMs · Rule complexity ladder Level 6accuracy: 75.9 -> 49.2Hurt
replace image with text-only FEN input (ablation)Qwen3-VL 235B · Predictive L6accuracy: 41.7 -> 79.0Improved
add chain-of-thought scratchpad (ablation)VLMs · Rule ladder L3-L6accuracy: e.g. GPT-5.2 Explicit L3 96.5 -> 81.3Hurt
add chain-of-thought scratchpad (ablation)GPT-5.2 · Predictive L5accuracy: 68.4 -> 82.1Improved
add chain-of-thought scratchpad (ablation)Qwen3-VL 235B · Predictive L6accuracy: 41.7 -> 23.1Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy