Research map / Vision-language and multimodal / Visual question answering
12-in-1: Multi-Task Vision and Language Representation Learning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Jointly train one shared model on 12 datasets rather than train independent task models | ViLBERT · 12 vision-and-language datasets | average score: 67.03 vs. 69.08; 2.05 points | Helped |
| Jointly train across task groups rather than within groups only | ViLBERT multi-task model · 12-task average (clean) | average score: 68.72 vs. 69.08 | Helped |
| Fine-tune task-specific models from the All-Tasks model | ViLBERT · 12-task average | average score: 67.25 vs. 70.24 | Helped |
| Use one task token per dataset instead of one per output head (ablation) | All-Tasks model · 12-task average | average score: 68.52 -> 69.08 | Helped |
| Add task tokens rather than remove them (ablation) | All-Tasks model · 12-task average | average score: 68.53 -> 69.08 | Helped |
| Use dynamic stop-and-go scheduling rather than no scheduling (ablation) | All-Tasks model · 12-task average | average score: 68.52 -> 69.08 | Helped |
| Use dynamic stop-and-go scheduling rather than curriculum training (ablation) | All-Tasks model · 12-task average | average score: 67.24 -> 69.08 | Helped |
| Use dynamic stop-and-go scheduling rather than anti-curriculum training (ablation) | All-Tasks model · 12-task average | average score: 67.98 -> 69.08 | Helped |
| Remove Conceptual Captions pretraining (ablation) | All-Tasks model · 12-task average | average score: 65.56 vs. 69.08 | Hurt |
| Remove verification task group G4 (ablation) | All-Tasks model · G1+G2+G3 average | average score: 67.39 vs. 67.96 | Helped |
| Jointly train with NLVR2 verification tasks alongside other tasks (ablation) | Representative multi-task models · Other task groups, especially retrieval | relative performance: -1.36%; -4.13% | Hurt |
| Use DSG iteration gap Δ=8 instead of Δ=4 (ablation) | All-Tasks model · 12-task average | average score: 69.08 -> 69.15 | Helped |
Closest papers by meaning
- On Scaling Up a Multilingual Vision and Language Model (CVPR 2024 · similarity 0.92)
- TaskGalaxy: Scaling Multi-modal Instruction Fine-tuning with Tens of Thousands Vision Task Types (ICLR 2025 · similarity 0.91)
- Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.91)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.90)
- Same or Not? Enhancing Visual Perception in Vision-Language Models (CVPR 2026 · similarity 0.90)
- Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.