Research map / Vision-language and multimodal / Multimodal LLMs

GeoDial: A Multimodal Dialog Tutoring Dataset for Geometry Problem-Solving with Visual Tutor Turns

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GeoDial, a 1,341-dialog geometry tutoring dataset with teacher diagram highlights, and benchmarks fine-tuned VLMs on generating tutor utterances and highlights.

Problem

Existing tutoring datasets are text-only, so AI tutors cannot teach with the visually grounded diagram cues human instructors use in geometry.

Approach

Collects teacher-student dialogs in which teachers select dialog acts, subacts, feedback types, and draw green diagram highlights that are standardized into lines, angles, arcs, and labels, with VLM-simulated students and VLM-suggested utterances; fine-tunes VLMs to jointly predict acts, highlights, and utterances.

Key results

• Fine-tuned Qwen3-VL-32B achieves the best act (46.5), subact (31.2) and feedback (72.1) Macro-F1 and BLEU 14.3 on the test set.
• Fine-tuning improves highlight prediction across the board but lowers Decision F1 for lines and angles, as models learn to abstain more often.
• Separating highlight prediction from utterance generation with weighted (positive-upsampled) training yields the best Combined F1 for lines (40.9) and angles (27.1).

Surprising findings

• Fine-tuning makes models abstain from highlighting more often, lowering Decision F1 for lines and angles despite better prediction.
• Highlight performance is much higher when generated from ground-truth utterances, which is not realizable at inference, highlighting coupling between visual actions and language.

Limitations

• Students are simulated by VLMs, which may not reflect real student behavior and should not be used to train student models.
• Highlights are standardized to lines, angles, arcs, and labels, losing some visual nuance.
• Only geometry is covered and the teacher interaction is one-way (students cannot modify diagrams).

Experimental setup

Datasets: Geometry3K PGPS9K GeoDial

Benchmarks: GeoDial test set

Metrics: Macro-F1 Highlight accuracy/F1 (Prediction) Decision F1 BLEU BERTScore

Baselines: Gemini 2.5 Flash GPT-4o Claude Sonnet 4 Qwen3 8B Qwen3 32B Llama 3.1 8B Qwen3-VL 8B Qwen3-VL 32B InternVL 8B Llama 3.2 11B Vision Random Turn

Builds on: MathDial Geometry3K PGPS9K

Closest papers by meaning

• Explain with Visual Keypoints Like a Real Mentor! A Benchmark for Multimodal Solution Explanation (AAAI 2026 · similarity 0.89)
• GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training (ICLR 2025 · similarity 0.88)
• GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning (CVPR 2026 · similarity 0.87)
• GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation (ICLR 2026 · similarity 0.87)
• Socratic-Geo: Synthetic Data Generation and Cross-Modal Geometric Reasoning via Multi-Agent Interaction (CVPR 2026 · similarity 0.87)
• Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency (NeurIPS 2026 · similarity 0.87)