Research map / Vision-language and multimodal / Multimodal LLMs

ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ReasonMap, a benchmark of 1,008 QA pairs over high-resolution transit maps from 30 cities for fine-grained visual spatial reasoning, evaluates 16 MLLMs with a two-level accuracy/map-score framework, and provides a GRPO reinforcement fine-tuning baseline on Qwen2.5-VL.

Problem

MLLMs' ability to jointly perform fine-grained visual perception and spatial reasoning/route planning over information-dense, high-resolution maps is underexplored, as existing benchmarks target either perception or coarse spatial reasoning.

Approach

A semi-automated pipeline builds transit-map route-planning questions with API-verified reference routes and difficulty labels, scored by a rule-based correctness check plus a map score that gives partial credit for route names, endpoints, and via stops; GRPO with accuracy and format rewards serves as a training baseline.

Key results

• OpenAI o3 achieves the best weighted accuracy of 63.02% on short questions and 59.11% on long questions, while among open-source models Qwen2.5-VL-72B-Instruct reaches 26.65% / 24.22%.
• Among open-source models, base variants outperform reasoning variants (e.g., Kimi-VL-A3B-Instruct 12.76% vs. Thinking 5.47% on short questions), whereas closed-source reasoning models (Doubao-415 43.14%) beat base ones (Doubao-115 34.20%).
• GRPO reinforcement fine-tuning raises Qwen2.5-VL-7B short-question weighted accuracy from 13.28% to 26.22% and long-question accuracy from 7.12% to 26.04%, with fewer tokens.

Surprising findings

• Among open-source models, reasoning-tuned variants underperform their base counterparts, the reverse of the trend seen in closed-source models.
• Qwen2.5-VL-3B-Instruct shows minimal or even improved performance when visual input is masked, suggesting reliance on prior knowledge rather than visual reasoning.

Limitations

• Closed-source model comparisons are not fully transparent, as the authors note that model details are undisclosed.
• Benchmark is limited to 30 transit maps and 1,008 QA pairs, with the hard-difficulty subset only 7.8% of the data.
• Annotation quality required correction: three experts identified an approximately 16% error rate before correction.

Experimental setup

Datasets: ReasonMap

Benchmarks: ReasonMap

Metrics: weighted accuracy map score token count

Baselines: Qwen2.5-VL-3B-Instruct Qwen2.5-VL-72B-Instruct InternVL3-78B Kimi-VL-A3B-Thinking QvQ-72B-Preview Skywork-R1V-38B OpenAI o3 Gemini-2.5-Flash Doubao-1.5-thinking-pro

Builds on: GRPO Qwen2.5-VL

Closest papers by meaning

• RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning (ICLR 2026 · similarity 0.92)
• ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.91)
• No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers (ICLR 2026 · similarity 0.90)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.90)
• Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.90)
• SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.89)