Research map / Vision-language and multimodal / Multimodal LLMs

GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GeoBench, a 1,021-sample hierarchical multiple-choice benchmark for geometric problem solving across six tasks in four van Hiele-inspired levels, generated with TrustGeoGen, and evaluates ~15 MLLMs and reasoning models.

Problem

Existing geometry benchmarks for VLMs rely on textbook problems (contamination risk), focus on final answers, and lack diagnostic granularity about which sub-skills (perception, planning, theorem use, backtracking) cause failures.

Approach

Generate formally verified synthetic geometry problems with TrustGeoGen and build six multiple-choice tasks mapped to four hierarchical reasoning levels, then correlate per-task accuracy with final-answer accuracy on solving problems and on OOD sets (GeoQA, Geometry3K).

Key results

• OpenAI-o3 reaches 91.00% on Sub-Goal Decomposition and 54.39% on Theorem Selection, the highest among evaluated models; Faulty Branch Localization stays near random (best 27.89% vs 25.71% random).
• Spearman correlation of Irrelevant Premise Filtering with GeoBench-solving final accuracy is 0.979, the highest among the six tasks.
• CoT prompting underperforms non-CoT on Faulty Branch Localization for the evaluated Qwen and GPT-4o models.

Surprising findings

• Chain-of-thought prompting systematically degrades performance on Faulty Branch Localization, possibly due to misleading erroneous reasoning chains in prompts.
• All evaluated MLLMs perform near random on Faulty Branch Localization, including the strongest reasoning models.

Limitations

• Restricted to planar (2D) geometry since TrustGeoGen targets 2D problems.
• Small sample sizes per task (e.g., 285 numeric-answer problems for correlation analysis) and correlations computed over a limited set of models.
• Synthetic problems rely on a fixed set of rules and constructions, which may limit diversity.

Experimental setup

Datasets: TrustGeoGen-generated geometry problems GeoQA Geometry3K OlympiadBench-Geo

Benchmarks: GeoBench GeoBench-solving GeoQA Geometry3K

Metrics: accuracy relative accuracy gain over random choice Spearman correlation

Baselines: GPT-4o GPT-4V OpenAI-o1 OpenAI-o3 Claude-3.7-sonnet Gemini-2.5-pro DeepSeek-R1 QvQ-72b CodePlot LLaVA-1.5 Qwen2-VL Qwen2.5-VL

Builds on: TrustGeoGen van Hiele model chain-of-thought prompting

Closest papers by meaning

• GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning (CVPR 2026 · similarity 0.91)
• Geoint-R1: Formalizing Multimodal Geometric Reasoning with Dynamic Auxiliary Constructions (CVPR 2026 · similarity 0.91)
• GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training (ICLR 2025 · similarity 0.91)
• Thinking with Geometry: Active Geometry Integration for Spatial Reasoning (ICML 2026 · similarity 0.90)
• Geometrically-Constrained Agent for Spatial Reasoning (CVPR 2026 · similarity 0.89)
• GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning (ICML 2026 · similarity 0.89)