Research map / Vision-language and multimodal / Visual question answering
Universal VQA models
Develops general and knowledge-based visual question answering with memory, reasoning transfer, and causal adapters.
100 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Visual question answering, keyword cluster “visual question · answering vqa · knowledge”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
21 papers in 2023–24 and 23 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 18.9% to 13.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Answer-Type Prediction for Visual Question Answering (CVPR 2016 · 132 citations)
- 2017 · The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions (CVPR 2017 · 97 citations)
- 2018 · Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering (CVPR 2018 · 5,159 citations)
- 2019 · Deep Modular Co-Attention Networks for Visual Question Answering (CVPR 2019 · 953 citations)
- 2020 · Counterfactual Samples Synthesizing for Robust Visual Question Answering (CVPR 2020 · 358 citations)
- 2021 · Roses Are Red, Violets Are Blue... but Should VQA Expect Them To? (CVPR 2021 · 85 citations)
- 2022 · An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA (AAAI 2022 · 284 citations)
- 2023 · Prompting Large Language Models With Answer Heuristics for Knowledge-Based Visual Question Answering (CVPR 2023 · 212 citations)
- 2024 · NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario (AAAI 2024 · 157 citations)
- 2025 · VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering (AAAI 2025 · 6 citations)
- 2026 · Cross-Granularity Hypergraph Retrieval-Augmented Generation for Multi-hop Question Answering (AAAI 2026 · 3 citations)
Earliest papers here
- Answer-Type Prediction for Visual Question Answering (CVPR 2016 · 132 citations)
- Community-Based Question Answering via Heterogeneous Social Network Learning (AAAI 2016 · 91 citations)
- Improving Recommendation of Tail Tags for Questions in Community Question Answering (AAAI 2016 · 17 citations)
What has been tried
88 papers here report 914 measured changes (a component added, removed or swapped, compared with a baseline): helped 74%, no clear effect 5%, hurt 18%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add detected box features | NuScenes-QA test · accuracy | 59.5 -> 58.9 | Hurt | NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario (AAAI 2024 · 157 citations) |
| Add BCA to VSA | EarthVQA · OA (%) | 72.55 -> 74.91 | Helped | EarthVQA: Towards Queryable Earth via Relational Reasoning-Based Remote Sensing Visual Question Answering (AAAI 2024 · 50 citations) |
| Use question-to-declaration transformation for 2D view selection | ScanQA val · EM@1 | 26.90 -> 26.98 | Helped | Bridging the Gap between 2D and 3D Visual Question Answering: A Fusion Approach for 3D VQA (AAAI 2024 · 20 citations) |
| add CKDM to LXMERT | COCO-QA · accuracy | 72.61 -> 74.94 | Helped | Object Attribute Matters in Visual Question Answering (AAAI 2024 · 9 citations) |
| Add chain-of-thought generation | VQA-X · CIDEr | CIDEr reduces by 1.2% on VQA-X | Helped | Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQA (AAAI 2024 · 9 citations) |
| add co-attention inference (CAIM) | Toronto COCO-QA, 5-way 1-shot · accuracy | 63.08 -> 65.94 | Helped | Cross-Modal Feature Distribution Calibration for Few-Shot Visual Question Answering (AAAI 2024 · 7 citations) |
| Rerank top 70 candidates instead of none | CIRR validation · Average | 82.06 -> 84.33 | Helped | VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering (AAAI 2025 · 6 citations) |
| add raw retrieved knowledge | OK-VQA · Score | 62.2 -> 65.3 | Helped | Notes-guided MLLM Reasoning: Enhancing MLLM with Knowledge and Visual Notes for Visual Question Answering (CVPR 2025 · 6 citations) |
| add adaptive visual prompts | VQA-RAD · open-set recall | 67.5 -> 71.9 | Helped | Fine-grained Adaptive Visual Prompt for Generative Medical Visual Question Answering (AAAI 2025 · 4 citations) |
| Add global text-text alignment to global inter-modality alignment | Average across VQA datasets · accuracy | 74.84 -> 75.28 | Helped | Alignment, Mining and Fusion: Representation Alignment with Hard Negative Mining and Selective Knowledge Fusion for Medical Visual Question Answering (CVPR 2025 · 3 citations) |
| Add processing to retrieval | InfoSeek validation, 10K samples · VQA accuracy | 34.22 -> 36.76 | Helped | Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering (NeurIPS 2025 · 3 citations) |
| use CTGR core features instead of Coarse2Fine features | GQA validation · accuracy | 66.7 -> 68.0 | Helped | Core-to-Global Reasoning for Compositional Visual Question Answering (AAAI 2025 · 2 citations) |
What reviewers will expect
Datasets and baselines named in the 14 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- SLAKE · 3 of 14 papers
- VQA-RAD · 3 of 14 papers
- OK-VQA · 3 of 14 papers
- GQA · 2 of 14 papers
- PathVQA · 2 of 14 papers
- VQA v2 · 2 of 14 papers
Baselines
- LXMERT · 2 of 14 papers
- PeFoMed · 2 of 14 papers
- M3AE · 2 of 14 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 44 papers; from OpenAlex and the papers' affiliations).
- Chinese Academy of Sciences · 3 papers
- Korea Advanced Institute of Science and Technology · 3 papers
- University of Chinese Academy of Sciences · 2 papers
- Hangzhou Dianzi University · 2 papers
- Harbin Institute of Technology · 2 papers
- Mohamed bin Zayed University of Artificial Intelligence · 2 papers
- Alibaba Group · 2 papers
- Amazon · 2 papers
- University of Michigan - Ann Arbor · 2 papers
- University of Cambridge · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Attention and prior-robust VQA (109 papers)
- Vision-language model VQA evaluation (56 papers)
- Multimodal LLM question answering (56 papers)
Elsewhere in the topic
- Multimodal question answering datasets (90 papers)
- Open-domain multimodal QA (73 papers)
- Visual reasoning diagnostics (62 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).