Research map / Vision-language and multimodal / Visual question answering
MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-Based Visual Question Answering
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MuKEA's multimodal knowledge extraction and accumulation framework instead of Knowledge is Power | KB-VQA · OK-VQA | accuracy: 39.24 -> 42.59 | Helped |
| Use MuKEA instead of MCAN with knowledge retrieval | KB-VQA · KRVQA overall | accuracy: 21.30 -> 27.38 | Helped |
| Add Triplet Consistency Loss (ablation) | MuKEA · OK-VQA | accuracy: 41.35 -> 42.59 | Helped |
| Add Semantic Consistency Loss (ablation) | MuKEA · OK-VQA | accuracy: 42.06 -> 42.59 | Helped |
| Add both Triplet Consistency Loss and Semantic Consistency Loss (ablation) | MuKEA · OK-VQA | accuracy: 40.84 -> 42.59 | Helped |
| Add TransE Loss (ablation) | MuKEA · OK-VQA | accuracy: 24.50 -> 42.59 | Helped |
| Use Gumbel-Softmax hard attention instead of soft attention for head extraction (ablation) | MuKEA · OK-VQA | accuracy: 40.67 -> 42.59 | Helped |
| Use the [CLS] token rather than self-attention over all output tokens for relation extraction (ablation) | MuKEA · OK-VQA | accuracy: 40.79 -> 42.59 | Helped |
| Learn the tail representation rather than use fixed GloVe embeddings (ablation) | MuKEA · OK-VQA | accuracy: 41.42 -> 42.59 | Helped |
| Include the head entity in the triplet (ablation) | MuKEA · OK-VQA | accuracy: 39.83 -> 42.59 | Helped |
| Include the relation in the triplet (ablation) | MuKEA · OK-VQA | accuracy: 39.40 -> 42.59 | Helped |
| Accumulate basic knowledge using VQA 2.0 pre-training (ablation) | MuKEA · OK-VQA | accuracy: 36.35 -> 42.59 | Helped |
Closest papers by meaning
- Object Attribute Matters in Visual Question Answering (AAAI 2024 · similarity 0.91)
- Question Aware Vision Transformer for Multimodal Reasoning (CVPR 2024 · similarity 0.90)
- BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining (AAAI 2024 · similarity 0.89)
- Multimodal Commonsense Knowledge Distillation for Visual Question Answering (Student Abstract) (AAAI 2025 · similarity 0.89)
- Separation of Powers: On Segregating Knowledge from Observation in LLM-enabled Knowledge-based Visual Question Answering (CVPR 2025 · similarity 0.89)
- Re-ranking Reasoning Context with Tree Search Makes Large Vision-Language Models Stronger (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.