Research map / Vision-language and multimodal / Multimodal LLMs
G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds Geo170K, a ~170K geometric image-caption and QA dataset generated from Geometry3K and GeoQA+ with text-only ChatGPT, and trains G-LLaVA (LLaVA-style, 7B/13B) that outperforms GPT-4V on MathVista geometry problems.
Problem
Multimodal LLMs hallucinate on geometric diagrams and struggle to understand basic geometric elements and relations, which limits their ability to solve geometry problems; existing geometry datasets are small and lack diagram descriptions.
Approach
Use text-only ChatGPT to synthesize geometry data from existing annotated datasets: inverse information recovery for captions, contrastive QA from human-labeled logic forms, and augmentation of QA pairs (equation solving, value scaling, reformulating conditions as unknown, paraphrase), then align and instruction-tune a LLaVA model on it.
Key results
- G-LLaVA-7B reaches 53.4% on MathVista GPS testmini, above GPT-4V (50.5%); G-LLaVA-13B reaches 56.7%.
- On GeoQA, G-LLaVA-13B achieves 67.0% top-1 accuracy versus 50.0% for UniMath and 46.8% for Geoformer; G-LLaVA-7B reaches 64.2%.
- Removing the alignment phase lowers G-LLaVA-7B from 64.2% to 62.8% on GeoQA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train a 7B LLaVA model with Geo170K geometric alignment and instruction-tuning data | LLaVA-7B · MathVista minitest GPS | accuracy: 50.5 -> 53.4 | Helped |
| Train a 13B LLaVA model with Geo170K geometric alignment and instruction-tuning data | LLaVA-13B · MathVista minitest GPS | accuracy: 56.3 -> 56.7 | Helped |
| Train with Geo170K geometric alignment and instruction-tuning data | LLaVA-7B · GeoQA test | Top-1 accuracy: 50.0 -> 64.2 | Helped |
| Train with Geo170K geometric alignment and instruction-tuning data | LLaVA-13B · GeoQA test | Top-1 accuracy | Helped |
| Add the cross-modal geometric alignment phase (ablation) | G-LLaVA-7B · GeoQA test | Top-1 accuracy: 62.8 -> 64.2 | Helped |
Surprising findings
- The GeoQA results compare top-1 accuracy against baselines' top-10 accuracy, and the paper notes its top-1 score exceeds some baselines' top-10 scores (e.g., NGS 56.9, DPE-GPS 62.7 for 7B) despite the metric difference.
- Text-only ChatGPT-generated diagram descriptions were used as the basis for training MLLMs to read geometric diagrams, suggesting hallucination on diagrams can be reduced via text-only synthetic data.
Limitations
- Training data is derived from ChatGPT-generated text and existing datasets, so diagram descriptions may contain errors or hallucinations.
- Evaluation is on a limited set of geometry benchmarks (MathVista GPS, GeoQA), with mixed top-1 vs top-10 comparisons to prior work.
- Alignment phase gives only a modest gain (about 1.4 points on GeoQA).
Experimental setup
Datasets: Geometry3K GeoQA+ GeoQA MathVista
Benchmarks: MathVista (GPS testmini) GeoQA
Metrics: accuracy top-1 accuracy
Baselines: GPT-4V Gemini Ultra Multimodal Bard LLaVA-13B LLaVA-7B Geoformer UniMath SCA-GPS DPE-GPS NGS
Builds on: LLaVA Geometry3K GeoQA+ MathVista
Closest papers by meaning
- Do Large Language Models Truly Understand Geometric Structures? (ICLR 2025 · similarity 0.92)
- GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training (ICLR 2025 · similarity 0.90)
- GNS: Solving Plane Geometry Problems by Neural-Symbolic Reasoning with Multi-Modal LLMs (AAAI 2025 · similarity 0.89)
- GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning (CVPR 2026 · similarity 0.89)
- MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts (CVPR 2025 · similarity 0.89)
- Geoint-R1: Formalizing Multimodal Geometric Reasoning with Dynamic Auxiliary Constructions (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.