Research map / Vision-language and multimodal / Visual question answering
Tips and Tricks for Visual Question Answering: Learnings From the 2017 Challenge
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use bottom-up region features instead of 7×7 ResNet features (ablation) | VQA model · VQA v2 validation | VQA Score (All): 59.35 $\pm$0.10 -> 63.15 $\pm$0.08 | Helped |
| Add Visual Genome training data (ablation) | Reference VQA model · VQA v2 validation | VQA Score (All): 62.48 $\pm$0.15 -> 63.15 $\pm$0.08 | Helped |
| Keep balanced question pairs in the same minibatches (ablation) | Reference VQA model · VQA v2 validation | Accuracy over balanced pairs: 34.54 -> 34.66 | Helped |
| Use 300-dimensional pretrained GloVe for question words (ablation) | Reference VQA model · VQA v2 validation | VQA Score (All): 62.28 $\pm$0.06 -> 63.15 $\pm$0.08 | Helped |
| Use answer-specific pretrained text and image output embeddings (ablation) | Reference VQA model · VQA v2 validation | VQA Score (All): 62.64 $\pm$0.15 -> 63.15 $\pm$0.08 | Helped |
| Use sigmoid rather than softmax answer outputs (ablation) | VQA model · VQA v2 validation | VQA Score (All): 60.47 $\pm$0.08 -> 63.15 $\pm$0.08 | Helped |
| Train on soft annotator scores rather than binary targets (ablation) | VQA model · VQA v2 validation | VQA Score (All): 54.34 $\pm$0.19 -> 63.15 $\pm$0.08 | Helped |
| Use gated tanh rather than ReLU or tanh activations (ablation) | VQA model · VQA v2 validation | VQA Score (All): 61.63 $\pm$0.21 -> 63.15 $\pm$0.08 | Helped |
| Replace one-layer forward GRU with bag-of-words averaging (ablation) | Question encoder · VQA v2 validation | VQA Score (All): 63.15 $\pm$0.08 -> 62.53 $\pm$0.09 | Hurt |
| Use adaptive bottom-up regions rather than fixed K=36 (ablation) | VQA model · VQA v2 validation | VQA Score (All): 62.82 $\pm$0.14 -> 63.15 $\pm$0.08 | Helped |
| Increase answer-vocabulary frequency threshold from >8 to >10 (ablation) | Output vocabulary · VQA v2 validation | VQA Score (All): 63.15 $\pm$0.08 -> 63.30 $\pm$0.06 | No clear effect |
| Use two attention heads rather than one (ablation) | Bottom-up attention VQA model · VQA v2 validation | VQA Score (All): 63.15 $\pm$0.08 -> 63.10 $\pm$0.05 | No clear effect |
Closest papers by meaning
- Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering (CVPR 2018 · similarity 0.94)
- The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions (CVPR 2017 · similarity 0.94)
- Self-Critical Reasoning for Robust Visual Question Answering (NeurIPS 2019 · similarity 0.93)
- Answer Them All! Toward Universal Visual Question Answering Models (CVPR 2019 · similarity 0.93)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.92)
- FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.