Research map / Vision-language and multimodal / Visual question answering
Supervising the Transfer of Reasoning Patterns in VQA
Corentin Kervadec, Christian Wolf, Grigory Antipov, Moez Baccouche, Madiha Nadri
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add program supervision to oracle transfer | VL-Transformer · GQA test-dev | accuracy: 57.0{ $\pm0.3$} -> 57.8{ $\pm0.2$} | Improved |
| Add program supervision to oracle transfer | VL-Transformer · GQA-OOD test | acc-tail: 48.2{ $\pm0.3$} -> 48.8}{ $\pm0.1$ | Improved |
| Add program supervision to oracle transfer with LXMERT pre-training | VL-Transformer · GQA-OOD test | acc-tail: 47.1 -> 48.0}{ $\pm0.6$ | Improved |
| Add full program supervision, especially visual-argument supervision (ablation) | Compact VL-Transformer · GQA val. | accuracy: 62.2 -> 66.2 | Improved |
| Add full program supervision, especially visual-argument supervision (ablation) | Compact VL-Transformer · GQA-OOD val. | acc-tail: 46.9 -> 49.9 | Improved |
| Add visual-argument supervision to the other program losses (ablation) | Compact VL-Transformer · GQA val. | accuracy: 63.7 -> 66.2 | Improved |
| Add visual-argument supervision to the other program losses (ablation) | Compact VL-Transformer · GQA-OOD val. | acc-tail: 47.3 -> 49.9 | Improved |
| Predict programs after cross-modal rather than uni-modal layers (ablation) | Oracle transfer VL-Transformer · GQA val. | accuracy: 66.5 -> 67.4 | Improved |
| Replace ground-truth program labels with programs from other questions (ablation) | Compact VL-Transformer · GQA val. | accuracy: 66.2 -> 61.4 | Hurt |
| Increase object proposals from 36 to 100 per image | Oracle transfer + program supervision · GQA test-dev | accuracy: 57.0{ $\pm0.4$} -> 58.2}{ $\pm0.1$ | Improved |
| Use VinVL instead of RCNN visual features | Oracle transfer + program supervision · GQA test-dev | accuracy: 58.2}{ $\pm0.1$ -> 60.9}{ $\pm0.2$ | Improved |
Closest papers by meaning
- How Transferable Are Reasoning Patterns in VQA? (CVPR 2021 · similarity 0.90)
- Query and Attention Augmentation for Knowledge-Based Explainable Reasoning (CVPR 2022 · similarity 0.88)
- Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding (NeurIPS 2018 · similarity 0.87)
- Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning" (ICML 2020 · similarity 0.86)
- Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answering (AAAI 2018 · similarity 0.86)
- Knowledge Acquisition for Visual Question Answering via Iterative Querying (CVPR 2017 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
