Research map / Vision-language and multimodal / Visual question answering
Question Type Guided Attention in Visual Question Answering
Yang Shi, Tommaso Furlanello, Sheng Zha, Animashree Anandkumar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add QTA weighting of image features (ablation) | CATL · TDIUC test | overall accuracy: 82.23 -> 83.62 | Improved |
| add QTA weighting of image features (ablation) | CATL (W) · TDIUC test | overall accuracy: 83.92 -> 85.03 | Improved |
| add QTA weighting of image features (ablation) | CATL (W) · TDIUC test | harmonic MPT: 57.83 -> 60.08 | Improved |
| add QTA weighting of image features (ablation) | CATL · TDIUC test | harmonic MPT: 55.95 -> 54.60 | Hurt |
| concatenate ResNet and Faster R-CNN features (ablation) | CAT1 · TDIUC test | overall accuracy: 72.44 -> 73.05 | Improved |
| use Faster R-CNN instead of ResNet features (ablation) | CAT1 with NMT · TDIUC test | overall accuracy: 71.41 -> 72.44 | Improved |
| replace pre-trained NMT text feature with jointly-trained LSTM (ablation) | CAT1 with ResNet · TDIUC test | overall accuracy: 71.41 -> 77.55 | Improved |
| add Word2Vec to LSTM text feature (ablation) | CAT1L Faster R-CNN · TDIUC test | overall accuracy: 78.66 -> 80.16 | Improved |
| add question type as plain embedding input (QT) instead of attention (QTA) (ablation) | CATL / CAT · TDIUC test | accuracy: 3-5% | Improved |
| add absurd-question removal in training (ablation) | CATL-QTA-M · TDIUC test | simple accuracy: 84.33 -> 80.95 | Mixed |
| add QTA to MCB model | MCB · TDIUC test | overall accuracy: 84.97 vs 85.03 | Hurt |
| predict question type with multi-task head instead of ground-truth type (ablation) | CATL-QTA (W) · TDIUC test | simple accuracy: 85.03 -> 84.33 | Hurt |
Closest papers by meaning
- An Analysis of Visual Question Answering Algorithms (ICCV 2017 · similarity 0.91)
- Multi-Level Attention Networks for Visual Question Answering (CVPR 2017 · similarity 0.91)
- Question-Guided Hybrid Convolution for Visual Question Answering (ECCV 2018 · similarity 0.91)
- Re-Attention for Visual Question Answering (AAAI 2020 · similarity 0.90)
- Visual Question Answering as a Meta Learning Task (ECCV 2018 · similarity 0.90)
- Multi-Question Learning for Visual Question Answering (AAAI 2020 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
