Research map / Vision-language and multimodal / Multimodal LLMs
MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MicroVQA, a 1,042-question expert-curated multiple-choice benchmark for multimodal research-level reasoning in biological microscopy, with a two-stage MCQ generation pipeline (DSPy-optimized prompting plus agent-based RefineBot) that removes language shortcuts.
Problem
Existing multimodal scientific reasoning benchmarks are college-level or emphasize low-level perception, leaving research-level multimodal reasoning (image understanding, hypothesis generation, experiment proposal) under-evaluated, and standard LLM-based MCQ generation induces language shortcuts.
Approach
Expert-authored long-form microscopy VQA samples are converted into MCQs via a two-stage pipeline: a DSPy-optimized exam-style prompt, then RefineBot, an evaluator/rewriter/checker agent loop that rewrites questions until text-only shortcuts are removed.
Key results
- Best model o1 reaches 52.8% overall accuracy on MicroVQA, versus a human expert baseline of 50.3%.
- RefineBot lowers relative accuracy across models by 35-42% from Stage 1 to Stage 2 MCQs (e.g., GPT-4o 79.7 to 46.8).
- LLaVA-Med-Mistral-7B outperforms its base LLaVA-Mistral-7B by 4.5 points overall (43.0 vs 39.8).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Stage 2 RefineBot after Stage 1 exam-style MCQ generation (ablation) | MicroVQA MCQ generation · MicroVQA | model accuracy: 79.7 -> 46.8; 75.6 -> 46.2; 82.2 -> 51.7; 77.3 -> 47.1; 81.6 -> 52.8; 80.1 -> 49.8; 79.1 -> 51.1 | Helped |
| Fine-tune LLaVA-Mistral-7B on scientific articles as LLaVA-Med | LLaVA-Mistral-7B · MicroVQA | accuracy: 39.8 -> 43.0 | Helped |
| Use smaller versions of model families instead of their larger equivalents | Gemini-1.5, QwenVL, VILA, and Llama · MicroVQA | accuracy: less than 3 | Mixed |
| Use a 7B Qwen-2-VL model instead of the 72B model | Qwen-2-VL · MicroVQA | accuracy: 47.5 -> 48.8 | Helped |
| Use a 13B VILA1.5 model instead of the 40B model | VILA1.5 · MicroVQA | accuracy: 47.5 -> 41.8 | Hurt |
| Evaluate GPT-4o without the image | GPT-4o · MicroVQA MCQ generation | accuracy: 93% | Hurt |
| Use Stage 2 RefineBot, including its generation models, during refinement (ablation) | GPT-4o and Claude-3.5 models · MicroVQA | accuracy relative drop: less than 10 points of relative drop | Hurt |
Surprising findings
- GPT-4o scores 93% on naive generated MCQs without the image, indicating language shortcuts.
- Smaller models lose little accuracy versus larger counterparts, suggesting multimodal rather than language reasoning is the bottleneck.
Limitations
- Relatively small dataset (1,042 questions) limits statistical power for subset comparisons.
- Evaluation uses multiple-choice format, not open-ended answers.
- RefineBot uses GPT-4o and Claude-3.5-Sonnet, introducing a small bias against those models; shortcuts cannot be fully guaranteed removed.
Experimental setup
Datasets: MicroVQA MicroBench ScienceQA MMMU MMMU-Pro
Benchmarks: MicroVQA
Metrics: accuracy
Baselines: o1 Claude-3.5-Sonnet Gemini-Pro-1.5 GPT-4o Qwen-2-VL-7b LLaVA-Med-Mistral-7B VILA1.5-40b Pixtral-Large
Builds on: MMMU GPQA DSPy Bloom's taxonomy
Closest papers by meaning
- MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models (AAAI 2026 · similarity 0.89)
- SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers (NeurIPS 2024 · similarity 0.89)
- LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.89)
- Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning (NeurIPS 2025 · similarity 0.89)
- SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models (ICML 2026 · similarity 0.89)
- Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.