Research map / Vision-language and multimodal / Multimodal LLMs

We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces We-Math 2.0, a system combining a 491-knowledge-point hierarchy, GeoGebra-rendered MathBook-Standard/Pro datasets with a 3D difficulty space, a two-stage SFT+RL training pipeline (MathBook-RL) on Qwen2.5-VL-7B, and the MathBookEval benchmark.

Problem

MLLMs struggle with complex visual mathematical reasoning, partly because existing datasets lack comprehensive knowledge coverage, model-centric difficulty modeling, and emphasis on reasoning generalization across subproblems and same-type problems.

Approach

Organizes math data via a five-level knowledge hierarchy with principle-level annotation, expands seeds into multi-image and multi-question variants, defines a three-axis difficulty space (step, visual, contextual) for curriculum RL with GRPO, using a mean-based reward over same-principle variant groups and knowledge/modality incremental scheduling.

Key results

• MathBook-7B reaches 48.7 average across four benchmarks, versus 42.6 for its Qwen2.5-VL-7B backbone, with 73.0 on MathVista and 48.4 on We-Math.
• MathBook-7B uses only 1K SFT + 9.8K RL samples and outperforms several RL baselines trained on 15K-260K+ samples on We-Math (48.4 vs. WeThink-7B 48.0).
• Ablation shows SFT alone yields 65.8 on MathVista, while adding both RL stages gives 73.0; pre-aligned RL alone lifts We-Math to 46.7.

Surprising findings

• Natural-language CoT SFT outperforms a structured step-wise CoT format, and scaling SFT from 1K to 10K did not consistently help.
• Models perform below 50% on problems requiring 7-10 knowledge points, and geometry is consistently weaker than algebra on MathBookEval.

Limitations

• Relies on GPT-4o for annotation, rewriting, and evaluation judging, which may introduce bias.
• Ablations are reported only for MathBook-7B built on Qwen2.5-VL-7B.
• Some compared methods use very different training data budgets, complicating fair comparison.

Experimental setup

Datasets: MathBook-Standard MathBook-Pro

Benchmarks: MathVista MathVision We-Math MathVerse MathBookEval

Metrics: accuracy strict score

Baselines: GPT-4o Gemini-1.5-Pro Qwen2.5-VL-7B InternVL2.5-8B URSA-8B MM-Eureka-7B WeThink-7B VLAA-Thinker-7B R1-OneVision-7B R1-VL-7B

Builds on: GRPO Qwen2.5-VL We-Math

Closest papers by meaning

• MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning (AAAI 2026 · similarity 0.92)
• Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoning (AAAI 2025 · similarity 0.92)
• MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts (CVPR 2025 · similarity 0.91)
• DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models (ICLR 2025 · similarity 0.91)
• Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency (NeurIPS 2026 · similarity 0.91)
• CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solving (ICLR 2026 · similarity 0.90)