Research map / Vision-language and multimodal / Multimodal LLMs

Enhancing Geometric Perception in VLMs via Translator-Guided Reinforcement Learning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GeoPerceive, a canonical DSL-based benchmark and auto data engine for geometric diagram perception, and GeoDPO, a translator-guided DPO method that scores VLM natural-language descriptions via an NL-to-DSL translator, improving perception over SFT.

Problem

VLMs misperceive basic geometric primitives and constraints in diagrams, and end-to-end GPS benchmarks conflate perception errors with reasoning errors; SFT on formal programs is sensitive to permutation-equivalent representations and distribution shift.

Approach

Use a canonical, one-to-one DSL to score diagram perception exactly, train an NL-to-DSL translator on synthetic NL-DSL pairs, and use translator-derived DSL-level F1 scores to rank sampled natural-language descriptions into DPO preference pairs so the VLM stays in the natural-language regime.

Key results

• On GeoPerceive Main-test, GeoDPO raises Qwen2.5-VL-7B overall score from 57.96 to 66.19 (+14.2%) and LLaVA-Next from 41.01 to 51.86 (+26.46%), outperforming SFT.
• On GeoPerceive-OOD, GeoDPO improves LLaVA-Next overall score from 42.48 to 45.88 (+8.0%) while SFT yields only +2.4%.
• On the MathVista geometry subset, InternVL3 accuracy rises from 29.06 to 40.39 with GeoDPO versus 30.05 with SFT.

Surprising findings

• SFT degrades constraint-level performance for InternVL3 (-6.3%) and LLaVA-Next on the in-domain test set.
• SFT yields regressions or negligible gains on the out-of-distribution perception set for several backbones.

Limitations

• Translator quality directly bounds reward quality; weaker translators (fewer training samples) degrade GeoDPO gains.
• Circle and compound-constraint recovery is notably weaker (circle F1 about 75%) and degrades with generation complexity.
• OOD evaluation uses only 100 hand-curated diagrams and the MathVista subset has 203 items, so statistics are small.

Experimental setup

Datasets: GeoPerceive translator split GeoPerceive main split MathVista geometry subset

Benchmarks: GeoPerceive Main-test GeoPerceive-OOD MathVista geometry multi-choice subset

Metrics: DSL-level precision recall F1 overall score accuracy

Baselines: SFT on DSL Raw pretrained VLM GeoPPO Qwen3-235B GPT-o3

Builds on: DPO LoRA AlphaGeometry Inter-GPS GeoDRL

Closest papers by meaning

• GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning (CVPR 2026 · similarity 0.89)
• GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation (ICLR 2026 · similarity 0.89)
• SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.88)
• GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training (ICLR 2025 · similarity 0.88)
• Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.88)
• Abstract 3D Perception for Spatial Intelligence in Vision-Language Models (CVPR 2026 · similarity 0.88)