Research map / Vision-language and multimodal / Visual question answering
Multi-Task Learning of Hierarchical Vision-Language Representation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Jointly train VQA with ICR and VG instead of training VQA alone (ablation) | Hierarchical vision-language model · VQA 2.0 validation | Accuracy: 65.50 -> 66.35 | Helped |
| Jointly train VQA with ICR (ablation) | Hierarchical vision-language model · VQA 2.0 validation | Accuracy: 65.50 -> 66.24 | Helped |
| Jointly train VQA with VG (ablation) | Hierarchical vision-language model · VQA 2.0 validation | Accuracy: 65.50 -> 65.85 | Helped |
| Jointly train MS-COCO ICR with VQA and VG (ablation) | Hierarchical vision-language model · MS-COCO ICR image annotation validation | R@1: 69.05 -> 70.43 | Helped |
| Jointly train MS-COCO ICR with VQA and VG (ablation) | Hierarchical vision-language model · MS-COCO ICR image retrieval validation | R@1: 56.47 -> 57.50 | Helped |
| Jointly train VG with VQA and ICR (ablation) | Hierarchical vision-language model · Flickr30k Entities validation | R@1: 58.09 -> 58.26 | Helped |
| Jointly train Flickr30k ICR with VQA and VG (ablation) | Hierarchical vision-language model · Flickr30k ICR image annotation validation | R@1: 67.16 -> 72.07 | Helped |
| Jointly train Flickr30k ICR with VQA and VG (ablation) | Hierarchical vision-language model · Flickr30k ICR image retrieval validation | R@1: 53.17 -> 56.42 | Helped |
| Jointly train VQA with Flickr30k ICR and VG rather than without ICR (ablation) | Hierarchical vision-language model · VQA 2.0 validation | Accuracy: 65.50 -> 66.09 | Helped |
| Jointly train VG with VQA and Flickr30k ICR (ablation) | Hierarchical vision-language model · Flickr30k Entities validation | R@1: 58.09 -> 58.03 | No clear effect |
| Use the proposed multi-task method rather than the strongest published single-task method | Flickr30k image annotation · Flickr30k image annotation | R@1: 55.5 -> 71.6 | Helped |
| Use the proposed multi-task method rather than the strongest published single-task method | VQA 2.0 · VQA 2.0 test-standard | Overall accuracy: 68.94 -> 69.57 | Helped |
Closest papers by meaning
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.87)
- Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Models (AAAI 2024 · similarity 0.87)
- Unified Generative and Discriminative Training for Multi-modal Large Language Models (NeurIPS 2024 · similarity 0.87)
- Interactive Visual Task Learning for Robots (AAAI 2024 · similarity 0.87)
- On Scaling Up a Multilingual Vision and Language Model (CVPR 2024 · similarity 0.87)
- MMRL: Multi-Modal Representation Learning for Vision-Language Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.