Research map / Vision-language and multimodal / Multimodal LLMs

GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GGBench, a 1,411-item tri-modal (text, GeoGebra code, image) benchmark for geometric construction as generative reasoning, evaluating end-to-end UMMs and code-generating LLMs/LRMs with GPT-4o-judged and human metrics.

Problem

Existing multimodal math benchmarks test discriminative answer selection or decoupled understanding and generation, not integrated generative reasoning where a model must construct a verifiable geometric solution from scratch.

Approach

Frame geometric construction as the testbed for generative reasoning: each item pairs natural-language construction steps with executable GeoGebra code and a rendered diagram, enabling code-execution and geometry-aware verification alongside VLM judging of plans, intermediate steps, and final figures.

Key results

• GPT-5 achieves the top VLM-I score of 57.08 and human score of 83.06, while the best end-to-end UMM Nano Banana reaches VLM-I 33.82.
• On code execution, GPT-5 attains Pass@1 of 79.02, followed by Claude Sonnet 4.5 (75.34) and DeepSeek-V3.1 (70.09).
• VLM-judge scores correlate with human ratings at Pearson r = 0.9295 over 100 samples per model.

Surprising findings

• Pixel-level metrics (PSNR, SSIM, LPIPS) show only weak correlation with geometric validity, as high perceptual similarity can mask structural errors.
• Code-similarity metrics such as BLEU or ROUGE-L do not predict executable success; Gemini 2.5 Pro has high textual similarity but lower Pass@1.

Limitations

• Dataset construction relies heavily on GPT-5 and Gemini 2.5 Pro generation with LLM-based filtering, which may introduce model-specific bias.
• Primary automatic metrics depend on GPT-4o as a VLM judge, with human validation on only 100 samples per model.
• Benchmark is restricted to GeoGebra-based geometric construction problems, limiting generality beyond geometry.

Experimental setup

Datasets: web-collected geometry problems

Benchmarks: GGBench GGBench-Code

Metrics: VLM-T VLM-I-Mid VLM-I-Res VLM-I LPIPS PSNR SSIM Pass@1 BLEU ROUGE-L

Baselines: Qwen-Image Seedream 4.0 Janus BAGEL Nano Banana GPT-4o GLM-4.5V Qwen3-VL GPT-4 GPT-5 Claude Sonnet 4.5 Gemini 2.5 Pro

Builds on: MathVista MathVerse MM-MATH MathOPEval

Closest papers by meaning

• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.92)
• DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models (ICML 2026 · similarity 0.91)
• Socratic-Geo: Synthetic Data Generation and Cross-Modal Geometric Reasoning via Multi-Agent Interaction (CVPR 2026 · similarity 0.90)
• GIR-Bench: Versatile Benchmark for Generating Images with Reasoning (ICLR 2026 · similarity 0.90)
• Exploring Spatial Intelligence from a Generative Perspective (CVPR 2026 · similarity 0.90)
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.90)