Research map / Vision-language and multimodal / Visual question answering
Weakly Supervised Grounding for VQA in Vision-Language Transformers
Aisha Urooj, Hilde Kuehne, Chuang Gan, Niels Da Vitoria Lobo, Mubarak Shah
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text-guided residual capsule connections (ablation) | Ours (C=32) · GQA val | pointing game accuracy (%): 29.81 -> 34.43 | Improved |
| add text-guided residual capsule connections (ablation) | Ours (C=32) · GQA val | overlap F1 (%): 19.37 -> 17.70 | Hurt |
| add residual capsule connections (ablation) | Ours (C=32) · GQA val | IOU F1 (%): 2.43 -> 2.47 | No clear effect |
| use 16 capsules instead of 48 (ablation) | Ours · GQA val | overlap F1 (%): 17.89 -> 24.84 | Improved |
| use 16 capsules instead of 48 (ablation) | Ours · GQA val | VQA accuracy (%): 53.65 -> 56.65 | Improved |
| initialize encoders with ViT and BERT weights (ablation) | Ours (C=32) · GQA val | VQA accuracy (%): 55.41 -> 58.86 | Improved |
| initialize encoders with ViT and BERT weights (ablation) | Ours (C=32) · GQA val | pointing game accuracy (%): 34.43 -> 27.06 | Hurt |
| add capsules and text-guided selection to backbone (open questions) (ablation) | backbone without capsules · GQA val open questions | overlap F1 (%): 30.70 -> 57.29 | Improved |
| add capsules and text-guided selection to backbone (compare questions) (ablation) | backbone without capsules · GQA val compare questions | overlap F1 (%): 13.27 -> 50.29 | Improved |
| cluster maximum points from all heads instead of using best single head (ablation) | Ours (C=16) · GQA val | pointing game accuracy (%): 23.08 -> 34.59 | Improved |
| add GQA samples to pre-training (ablation) | Ours (C=32) · GQA val | pointing game accuracy (%): 37.04 -> 34.43 | Hurt |
| add capsule grounding model versus no-caps on VQA accuracy (ablation) | backbone without capsules · GQA val | VQA accuracy (%): 62.43 -> 56.65 | Hurt |
Closest papers by meaning
- Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsules (CVPR 2021 · similarity 0.93)
- Can I Trust Your Answer? Visually Grounded Video Question Answering (CVPR 2024 · similarity 0.89)
- Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy (ICLR 2024 · similarity 0.89)
- Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models (ICLR 2024 · similarity 0.89)
- Nüwa: Mending the Spatial Integrity Torn by VLM Token Pruning (ICLR 2026 · similarity 0.89)
- Rethinking Token Reduction for Large Vision-Language Models (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
