Research map / Vision-language and multimodal / Visual question answering
VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answering
Yanan Wang, Michihiro Yasunaga, Hongyu Ren, Shinya Wada, Jure Leskovec
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VQA-GNN on top of RESERVE-L | RESERVE-L · VCR test | Q->AR accuracy: 72.0 -> 74.3 | Improved |
| replace SGEITL+VLBERT with VQA-GNN | SGEITL+VLBERT · VCR test | Q->AR accuracy: 59.6 -> 62.8 | Improved |
| use two modality-specialized GNNs instead of a single GNN (ablation) | VQA-GNN · VCR val (Q->A) | accuracy: 73.0 -> 75.1 | Improved |
| use two modality-specialized GNNs instead of single GNN with cross-modal edges (ablation) | VQA-GNN · VCR val (Q->A) | accuracy: 70.6 -> 75.1 | Improved |
| remove direct cross-modal edges in a single GNN (ablation) | single GNN · VCR val (Q->A) | accuracy: 70.6 -> 73.0 | Improved |
| add QA-concept node p (ablation) | scene-graph + concept-graph · VCR val (Q->A) | accuracy: 75.1 -> 77.1 | Improved |
| add scene graph to node z and p (ablation) | Node z · VCR val (Q->A) | accuracy: 53.8 -> 73.7 | Improved |
| add concept graph to node z (ablation) | Node z · VCR val (Q->A) | accuracy: 53.8 -> 69.0 | Improved |
| use bidirectional fusion instead of unidirectional fusion (ablation) | VQA-GNN · GQA val | accuracy: 86.3 -> 90.3 | Improved |
| use bidirectional fusion instead of average pooling (ablation) | VQA-GNN · GQA val | accuracy: 62.3 -> 90.3 | Improved |
| interconnect visual and textual scene graphs (ablation) | VQA-GNN · GQA val | accuracy: 87.9 -> 90.3 | Improved |
| replace GCN with VQA-GNN (textual SG) | GCN · GQA val | accuracy: 85.7 -> 87.9 | Improved |
Closest papers by meaning
- Multimodal Learning and Reasoning for Visual Question Answering (NeurIPS 2017 · similarity 0.94)
- Multimodal Commonsense Knowledge Distillation for Visual Question Answering (Student Abstract) (AAAI 2025 · similarity 0.92)
- MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-Based Visual Question Answering (CVPR 2022 · similarity 0.92)
- BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining (AAAI 2024 · similarity 0.91)
- Object Attribute Matters in Visual Question Answering (AAAI 2024 · similarity 0.91)
- Question Aware Vision Transformer for Multimodal Reasoning (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
