Research map / Vision-language and multimodal / Multimodal LLMs

Explain with Visual Keypoints Like a Real Mentor! A Benchmark for Multimodal Solution Explanation

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces the multimodal solution explanation task and the ME2 benchmark (1,000 Korean math geometry/graph problems with visual keypoints and keypoint-referencing explanations), finding that current MLLMs struggle to identify keypoints and, especially open-source models, to generate keypoint-grounded explanations.

Problem

Current MLLMs generate textual math solutions but do not identify or reference the visual keypoints (auxiliary lines, points, angles, marks) that human tutors use to explain solutions, and no benchmark evaluates this for educational explanation.

Approach

Defines a two-subtask benchmark: multiple-choice visual keypoint identification (with distractors built from similar problems) and keypoint-based explanation generation, with keypoints annotated via GPT-4o-assisted human-AI annotation and judged by GPT-4o and humans.

Key results

• On visual keypoint identification, Gemini 2.0 Flash reaches 0.576 overall accuracy, versus 0.536 for Qwen2.5-VL 72B and 0.200 for Math-PUMA (near chance).
• On LLM-judged keypoint-based explanation, Gemini 2.0 Flash scores 3.849 correctness, 3.489 fidelity, 4.103 referencing (1-5 scale), while Qwen2.5-VL 7B scores 3.005/2.375/3.132.
• Only 23.1% of Gemini 2.0 Flash cases succeed on both problem-solving and keypoint identification, and 10.5% for Qwen2.5-VL 7B.

Surprising findings

• Math-specialized models underperform generalist models on the solution recognition toy task, likely due to weakened instruction following.
• GPT-4o performs similarly to open-source models on solution recognition (0.259 overall) despite strong keypoint scores.

Limitations

• Data is in-house, Korean-origin, and limited to middle-to-high school geometry and graph problems.
• Keypoint identification is multiple-choice and the correct answer and solution summary are given, so it does not test open-ended keypoint generation.
• Single-image problems only; annotations partly generated with GPT-4o.

Experimental setup

Datasets: ME2 (in-house Korean math problems)

Benchmarks: ME2

Metrics: accuracy Likert correctness Likert fidelity Likert referencing BLEU-2 BLEU-4 ROUGE-L METEOR BERTScore Spearman correlation

Baselines: Molmo LLaVA-1.6 Qwen2-VL Qwen2.5-VL Math-PUMA URSA Math-LLaVA GPT-4o Gemini 2.0 Flash

Builds on: MathVista

Closest papers by meaning

• MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts (CVPR 2025 · similarity 0.90)
• GeoDial: A Multimodal Dialog Tutoring Dataset for Geometry Problem-Solving with Visual Tutor Turns (NeurIPS 2026 · similarity 0.89)
• VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs (ICLR 2026 · similarity 0.88)
• Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency (NeurIPS 2026 · similarity 0.88)
• ViRC: Enhancing Visual Interleaved Mathematical CoT with Reason Chunking (CVPR 2026 · similarity 0.88)
• Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset (NeurIPS 2024 · similarity 0.87)