Research map / Vision-language and multimodal / Visual question answering
Multimodal Learning and Reasoning for Visual Question Answering
Ilija Ilievski, Jiashi Feng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add scene classification module (ablation) | VQA baseline · VQA v2.0 val | overall accuracy: 55.13 -> 56.80 | Improved |
| add object detection module (ablation) | VQA+Sc · VQA v2.0 val | overall accuracy: 56.80 -> 58.46 | Improved |
| add object classification module (ablation) | VQA+Sc+oDec · VQA v2.0 val | overall accuracy: 58.46 -> 59.82 | Improved |
| add face detection module (ablation) | VQA+Sc+oDec+oCls · VQA v2.0 val | Other accuracy: 54.47 -> 53.79 | Hurt |
| add face detection module (ablation) | VQA+Sc+oDec+oCls · VQA v2.0 val | Yes/No accuracy: 72.88 -> 74.21 | Improved |
| add face analysis module (ablation) | VQA+...+fDec · VQA v2.0 val | overall accuracy: 60.35 -> 60.60 | Improved |
| add face analysis module (ablation) | VQA+...+fDec · VQA v2.0 val | Other accuracy: 53.79 -> 54.81 | Improved |
| add face analysis module (ablation) | VQA+...+fDec · VQA v2.0 val | Yes/No accuracy: 74.21 -> 73.78 | Hurt |
| replace Hadamard product with bilinear interaction (ablation) | ReasonNet · VQA v2.0 val | overall accuracy: 58.37 -> 60.60 | Improved |
| replace MCB with bilinear interaction (ablation) | ReasonNet · VQA v2.0 val | overall accuracy: 58.78 -> 60.60 | Improved |
| Hadamard product vs MCB fusion (ablation) | ReasonNet · VQA v2.0 val | overall accuracy: 58.37 -> 58.78 | No clear effect |
| ReasonNet vs previous best (MLB) | MLB · VQA v1.0 test-standard | overall accuracy: 65.1 -> 67.9 | Improved |
Closest papers by meaning
- VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answering (ICCV 2023 · similarity 0.94)
- MUREL: Multimodal Relational Reasoning for Visual Question Answering (CVPR 2019 · similarity 0.92)
- An Analysis of Visual Question Answering Algorithms (ICCV 2017 · similarity 0.92)
- Question Aware Vision Transformer for Multimodal Reasoning (CVPR 2024 · similarity 0.91)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.91)
- Object Attribute Matters in Visual Question Answering (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
