Research map / Vision-language and multimodal / Multimodal LLMs

MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Supplementary material for MDK12-Bench, a multimodal K-12 reasoning benchmark, describing a dynamic bootstrapping evaluation pipeline (text/visual transformations) plus additional analyses across difficulty, year, subject and language.

Problem

Existing MLLM benchmarks are limited in scale, coverage, and structure, and offer static, undifferentiated evaluation; the supplement elaborates how MDK12-Bench is evaluated and analyzed.

Approach

Augments each question-image-answer sample via judged textual transformations (word substitution, paraphrasing, question-type conversion) and visual transformations (expansion, color shift, style transfer) before scoring model responses with exact-match and partial-credit scoring.

Key results

• QVQ-72B reaches about 64.8% on easy 2016 questions versus 42.0% on easy 2025 questions, showing a decline on newer easy items.
• Gemini-2-flash scores 65.7% on easy 2025 questions and 56.4% on hard 2025 questions.
• Qwen2.5-VL-7B evaluated on Chinese translation shows minimal gap on easy and medium questions but a noticeable drop on hard questions.

Surprising findings

• For easy questions, model accuracy generally declines for more recent years, while medium and hard accuracy remains relatively stable across years.
• Knowledge-point injection yields large gains on easy questions but only moderate gains on medium and hard questions.

Limitations

• Hard subject-specific questions remain challenging, with all evaluated models producing incorrect answers in case studies.
• Translation-based bilingual evaluation shows performance drop on hard questions, attributed to formula and domain complexity.

Experimental setup

Datasets: MDK12-Bench

Benchmarks: MDK12-Bench

Metrics: accuracy partial-credit score

Baselines: QVQ-72B Qwen2.5-VL-72B GPT-4o Gemini-2-flash Gemini2-thinking

Closest papers by meaning

• MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models (AAAI 2026 · similarity 0.93)
• RBench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation (ICML 2025 · similarity 0.92)
• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.92)
• SEED-Bench: Benchmarking Multimodal Large Language Models (CVPR 2024 · similarity 0.92)
• MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (ICML 2025 · similarity 0.91)
• MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation (ICLR 2025 · similarity 0.91)