AI Paper Atlas

Research map / Vision-language and multimodal / Visual question answering

Weakly Supervised Grounding for VQA in Vision-Language Transformers

Aisha Urooj, Hilde Kuehne, Chuang Gan, Niels Da Vitoria Lobo, Mubarak Shah

ECCV 2022 · Vision-language and multimodal › Visual question answering · approach: Vision-language model VQA evaluation · 19 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add text-guided residual capsule connections (ablation)Ours (C=32) · GQA valpointing game accuracy (%): 29.81 -> 34.43Improved
add text-guided residual capsule connections (ablation)Ours (C=32) · GQA valoverlap F1 (%): 19.37 -> 17.70Hurt
add residual capsule connections (ablation)Ours (C=32) · GQA valIOU F1 (%): 2.43 -> 2.47No clear effect
use 16 capsules instead of 48 (ablation)Ours · GQA valoverlap F1 (%): 17.89 -> 24.84Improved
use 16 capsules instead of 48 (ablation)Ours · GQA valVQA accuracy (%): 53.65 -> 56.65Improved
initialize encoders with ViT and BERT weights (ablation)Ours (C=32) · GQA valVQA accuracy (%): 55.41 -> 58.86Improved
initialize encoders with ViT and BERT weights (ablation)Ours (C=32) · GQA valpointing game accuracy (%): 34.43 -> 27.06Hurt
add capsules and text-guided selection to backbone (open questions) (ablation)backbone without capsules · GQA val open questionsoverlap F1 (%): 30.70 -> 57.29Improved
add capsules and text-guided selection to backbone (compare questions) (ablation)backbone without capsules · GQA val compare questionsoverlap F1 (%): 13.27 -> 50.29Improved
cluster maximum points from all heads instead of using best single head (ablation)Ours (C=16) · GQA valpointing game accuracy (%): 23.08 -> 34.59Improved
add GQA samples to pre-training (ablation)Ours (C=32) · GQA valpointing game accuracy (%): 37.04 -> 34.43Hurt
add capsule grounding model versus no-caps on VQA accuracy (ablation)backbone without capsules · GQA valVQA accuracy (%): 62.43 -> 56.65Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy