Research map / Vision-language and multimodal / Multimodal LLMs
LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?
Xiaohan Wang, Mingze Yin, Yilin Zhao, sinbadliu sinbadliu, Dian Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| switch from parsed text/image input to full exam-page image input | Claude-opus-4.6 · Complex Layout Set | accuracy: 87.2 -> 55.0 | Hurt |
| switch from parsed input to full exam-page image input | Gemini-3-pro · Complex Layout Set | accuracy: 88.3 -> 60.0 | Hurt |
| switch from parsed input to full exam-page image input | GPT-4o · Complex Layout Set | accuracy: 35.0 -> 5.5 | Hurt |
| switch from parsed input to full exam-page image input | GPT series · Complex Layout Set | accuracy and OCS: average drop of 33.7% in Acc and 44.9 points in OCS | Hurt |
| evaluate process quality in addition to final answer | GPT-5 · Rigorous Process Set | OCS vs PES: 83.0 -> 56.0 | Hurt |
| evaluate process quality in addition to final answer | Gemini-3-pro · Rigorous Process Set | OCS vs PES: 90.9 -> 76.9 | Hurt |
| compare long-horizon subset with overall dataset | all models · Long-horizon Reasoning Set | ARL: -8 to -15 | Hurt |
| restrict generation budget to 10% of default length | most models · Long-horizon Reasoning Set | accuracy: approximately half of unconstrained accuracy | Hurt |
| use multi-LLM arbitration instead of a single GPT-5 judge | process-error evaluator · Rigorous Process Set sample | Cohen's kappa: 0.82 -> 0.87 | Improved |
| use multi-LLM arbitration instead of a single GPT-5 judge | process-error evaluator · Rigorous Process Set sample | accuracy: 82.0 -> 89.6 | Improved |
| compare subjects: Chemistry/Biology versus Mathematics | all models · LiveK12Bench | accuracy: all models perform worse in Chemistry and Biology compared to Mathematics and Physics | Hurt |
Closest papers by meaning
- EduDiag: A Benchmark for Educational Diagnostic Reasoning with Error Tracing and Correction on Large Multimodal Models (CVPR 2026 · similarity 0.90)
- BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset (NeurIPS 2025 · similarity 0.88)
- MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning (ICLR 2026 · similarity 0.88)
- RBench-V: A Primary Assessment for Visual Reasoning Models with Multimodal Outputs (NeurIPS 2025 · similarity 0.88)
- LiveBench: A Challenging, Contamination-Limited LLM Benchmark (ICLR 2025 · similarity 0.88)
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
