Research map / Vision-language and multimodal / Visual question answering
A Diagram is Worth a Dozen Images
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use sequential LSTM-based DPG inference instead of A* search | DSDP-Net · AI2D diagram parsing (test) | JIG score: 41.02 -> 51.45 | Improved |
| use sequential LSTM-based DPG inference instead of greedy search | DSDP-Net · AI2D diagram parsing (test) | JIG score: 28.96 -> 51.45 | Improved |
| use DPG-based attention (DQA-Net) instead of LSTM Q+I baseline, both trained on AI2D | diagram QA · AI2D diagram QA (test) | accuracy: 32.90 -> 38.47 | Improved |
| train the VQA baseline on AI2D instead of VQA | LSTM Q+I baseline · AI2D diagram QA (test) | accuracy: 29.06 -> 32.90 | Improved |
| use RF blob proposals instead of Edge Boxes | constituent proposals · AI2D blob proposals | average precision: 0.02 -> 0.7829 | Improved |
| use single-character localizer plus OCR instead of Tesseract end-to-end | text detection · AI2D text localization | precision: 0.2 -> 0.89 | Improved |
| use single-character localizer plus OCR instead of Tesseract end-to-end | text detection · AI2D text localization | recall: 0.46 -> 0.75 | Improved |
Closest papers by meaning
- CoG-DQA: Chain-of-Guiding Learning with Large Language Models for Diagram Question Answering (CVPR 2024 · similarity 0.90)
- Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs (ICLR 2026 · similarity 0.87)
- Do Vision-Language Models Really Understand Visual Language? (ICML 2025 · similarity 0.86)
- Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models (ICML 2025 · similarity 0.85)
- ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.85)
- DiGraphHal-Bench: Evaluating Multimodal Large Language Models on Complex Directed Graphs (CVPR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
