Research map / Vision-language and multimodal / Multimodal LLMs
LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?
Kexian Tang, Junyao Gao, Yanhong Zeng, Haodong Duan, Sun Yanan, Zhening Xing, Wenran Liu, Kai Chen, Kaifeng Lyu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| raise required reasoning steps k from 1 to 8 in PLAN-k-Step-VQA | GPT-5 · PLAN-k-Step-VQA | exact match accuracy: 90% -> 0% | Hurt |
| move from Next-Step to Ordering task | GPT-5 and most MLLMs · LEGO-Puzzles Elementary | accuracy: e.g. Qwen3-VL-32B 62.0 -> 34.0 | Hurt |
| proprietary vs open-source MLLMs | MLLMs · LEGO-Puzzles Elementary | accuracy: 72.0 (GPT-5) vs 57.0 (best open-source) | Improved |
| MLLMs vs human experts | best MLLMs · LEGO-Puzzles-Lite | accuracy: 71.8 vs 93.6 human | Hurt |
| change output from multiple choice to image generation | GPT-5 · LEGO-Puzzles planning | accuracy: 0 correct generative trajectories | Hurt |
| use LEGO-Puzzles Height/Adjacency vs 3DSRBench | proprietary MLLMs · 3DSRBench correlation | Pearson correlation: PCC 0.93 (Height), 0.98 (Adjacency) | Improved |
| Height task on MLLMs | MLLMs · LEGO-Puzzles Height | accuracy: 21/29 models below random threshold | Hurt |
| scale model size within families | Qwen2.5-VL · LEGO-Puzzles Elementary | accuracy: 31.5 -> 46.6 (7B -> 72B) | Improved |
Closest papers by meaning
- SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence (CVPR 2026 · similarity 0.91)
- Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.91)
- EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs (CVPR 2026 · similarity 0.91)
- Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.91)
- MineAnyBuild: Benchmarking Spatial Planning for Open-world AI Agents (NeurIPS 2025 · similarity 0.91)
- From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs (CVPR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
