Research map / Vision-language and multimodal / Visual question answering
Aligned Image-Word Representations Improve Inductive Transfer Across Vision-Language Tasks
Tanmay Gupta, Kevin Shih, Saurabh Singh, Derek Hoiem
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Genome classifiers jointly trained with VQA (ablation) | VQA model · VQA val | overall accuracy: 62.9 -> 64.1 | Improved |
| use shared word representations for visual classification instead of a standard classification layer (ablation) | joint VQA + Genome model · VQA val | overall accuracy: 64.1 -> 65.3 | Improved |
| add shared vision-language representation with Genome supervision (ablation) | VQA model · VQA val, what color | accuracy: 53.5 -> 62.1 | Improved |
| add shared representation with Genome supervision (ablation) | VQA model · VQA val, number | accuracy: 39.2 -> 38.8 | No clear effect |
| add shared representation with Genome supervision (ablation) | VQA model · VQA val, why/how | accuracy: 33.8 -> 33.6 | Mixed |
| add relevance features (ablation) | VQA + Genome (Shared) · VQA val | overall accuracy: 65.3 -> 65.5 | Improved |
| add binary features for answers (ablation) | VQA + Genome (Shared) with relevance features · VQA val | overall accuracy: 65.5 -> 65.7 | Improved |
| pretrain classifiers on Genome before joint training instead of training jointly from scratch (ablation) | VQA + Genome (Shared) · VQA val | overall accuracy: 65.3 -> 63.7 | Hurt |
| train with high weight on recognition losses and VQA as weak supervision | joint model · Visual Genome test | object classification accuracy | Improved |
Closest papers by meaning
- Bridging Vision and Language Spaces with Assignment Prediction (ICLR 2024 · similarity 0.90)
- Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy (ICLR 2024 · similarity 0.88)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.88)
- Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models (ICLR 2024 · similarity 0.88)
- Probing Visual Language Priors in VLMs (ICML 2025 · similarity 0.88)
- Revisiting the Role of Language Priors in Vision-Language Models (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
