Research map / Vision-language and multimodal / Multimodal LLMs

Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VCBench, a 1,720-question multi-image benchmark of grade 1-6 elementary math visual reasoning problems, and evaluates 26 LVLMs in multi-image and single-image settings, finding the best models stay below 50% accuracy versus 93.3% for humans.

Problem

Existing visual math benchmarks are knowledge-centric and do not isolate core vision-centric reasoning over multiple images, such as temporal, spatial, geometric, logical, and pattern reasoning with explicit visual dependencies.

Approach

A vision-centric, multi-image, multiple-choice benchmark built from Chinese elementary math textbooks (grades 1-6) that requires integrating 2 to 18 images per question, with a single-image layout control to test cross-image integration.

Key results

• Human average accuracy is 93.30% versus 29.83% for random guessing; best LVLMs remain below 50% (Qwen2.5-VL-72B-Instruct 48.08%, LLaVA-OneVision-72B 47.67%).
• Most models score higher in single-image layout than multi-image (average improvement +42.3%), e.g. Qwen-VL-Max +21.3%, Emu2-Chat +281.5%.
• CoT gives mixed effects: Qwen-VL-Max rises from 47.03 to 49.48 average and GPT-4o from 40.29 to 42.03.

Surprising findings

• Specialized multi-image model LLaVA-Interleave-7B performs better in multi-image than single-image setting (-17.6%).
• Questions annotated as hard yield higher accuracy than easy and medium questions for most models.

Limitations

• Multiple-choice format may not reflect free-form answering.
• Source limited to Chinese elementary textbooks translated into English via LLMs.
• Error analysis performed manually on only four models.

Experimental setup

Benchmarks: VCBench

Metrics: accuracy

Baselines: GPT-4o GPT-4o-mini Claude-3.7-Sonnet Gemini2.0-Flash Qwen-VL-Max Qwen2.5-VL-72B-Instruct LLaVA-OneVision-72B QVQ-72B-Preview InternVL2.5-78B Random Guess

Builds on: Chain-of-thought prompting MathVista MathVision

Closest papers by meaning

• MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts (CVPR 2025 · similarity 0.94)
• Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset (NeurIPS 2024 · similarity 0.92)
• DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models (ICLR 2025 · similarity 0.92)
• RBench-V: A Primary Assessment for Visual Reasoning Models with Multimodal Outputs (NeurIPS 2025 · similarity 0.92)
• VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs (ICLR 2026 · similarity 0.91)
• MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning (AAAI 2026 · similarity 0.91)