Research map / Vision-language and multimodal / Multimodal LLMs

MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MathSE, an iterative self-evolving framework for MLLM math reasoning that combines GPT-4o distilled SFT, a specialized outcome reward model (ORM) giving error-step analysis, and GPT-4o reflection on ORM-flagged errors, reporting gains on MathVista, MathVerse, MathVision and MathVL-test.

Problem

Fine-tuning MLLMs on static, teacher-distilled math datasets limits adaptation to novel or harder problems and lacks iterative depth for robust mathematical reasoning.

Approach

Iteratively fine-tune the student on its own verified correct reasoning paths, and convert ORM-flagged incorrect paths (with error step and analysis) into GPT-4o reflections that are added back into training data.

Key results

• On MathVL-test, MathSE-InternVL reaches 65.13% accuracy versus 52.25% for QVQ-72B, and MathSE-CogVLM2 reaches 64.70% from a 30.85% base.
• Average score on MathVista/MathVerse/MathVision rises from 42.58 to 50.62 for InternVL2.5-8B and from 29.86 to 45.77 for CogVLM2.
• Self-evolving data (62.35%) outperforms a fully GPT-4o-generated 240K set (58.00%) on MathVL-test with CogVLM2.

Surprising findings

• The ORM's binary judgments reach 97.10% overall accuracy on ORM-2K, with the error-analysis variant outperforming binary-only training on positive accuracy (94.20% vs 85.40%).

Limitations

• Reliance on GPT-4o for initial distillation and reflection makes the pipeline dependent on a closed-source teacher.
• Reasoning errors and question misunderstanding remain the dominant error types on MathVL-test.
• The ORM's error analyses come from GPT-4o-generated annotations, and the reflection prompt notes that the provided error steps may be unreliable.

Experimental setup

Datasets: MathVL GeoQA+ Geometry3K ChartQA UniGEO-Calculation MultiMath MAVIS Math-PUMA MathV360K

Benchmarks: MathVL-test MathVista MathVerse MathVision

Metrics: accuracy

Baselines: GPT-4o Claude 3.5 Sonnet Gemini-1.5-pro QVQ-72B Qwen2.5-VL-7B URSA-8B MultiMath-7B MAVIS-7B Math-LLaVA-13B G-LLaVA-13B

Builds on: chain-of-thought reflexion react

Closest papers by meaning

• We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning (ICLR 2026 · similarity 0.92)
• MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts (CVPR 2025 · similarity 0.92)
• Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency (NeurIPS 2026 · similarity 0.91)
• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.91)
• Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoning (AAAI 2025 · similarity 0.90)
• DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models (ICLR 2025 · similarity 0.90)