Research map / Vision-language and multimodal / Cross-modal learning
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text-only pre-training stage after image-only (ablation) | VLMo-Base · NLVR2 dev | accuracy: 80.33 -> 82.09 | Helped |
| add text-only pre-training stage (ablation) | VLMo-Base · Flickr30k TR | avg recall: 95.60 -> 95.67 | No clear effect |
| use MoME instead of standard Transformer (ablation) | VLMo · NLVR2 dev | accuracy: 78.81 -> 80.13 | Helped |
| add vision-language expert in top layers (ablation) | MoME · NLVR2 dev | accuracy: 79.58 -> 80.13 | Helped |
| add image-text matching objective (ablation) | VLMo (ITC+MLM) · NLVR2 dev | accuracy: 73.91 -> 80.13 | Helped |
| add masked language modeling objective (ablation) | VLMo (ITC+ITM) · NLVR2 dev | accuracy: 76.46 -> 80.13 | Helped |
| global instead of local hard negative mining (ablation) | VLMo · NLVR2 dev | accuracy: 77.70 -> 79.54 | Helped |
| share self-attention across modalities (ablation) | MoME · NLVR2 dev | accuracy: 78.92 -> 80.13 | Helped |
| VLMo dual encoder vs ALBEF fusion reranker | base-size VL models · Flickr30K TR | R@1: 94.3 -> 92.3 | Hurt |
| VLMo vs ALBEF | base-size VL models · VQA test-dev | vqa-score: 74.54 -> 76.64 | Helped |
| scale pre-training to 1B noisy pairs and larger batch | VLMo-Large · VQA test-dev | vqa-score: 79.94 -> 82.88 | Helped |
Closest papers by meaning
- Unveiling Encoder-Free Vision-Language Models (NeurIPS 2024 · similarity 0.90)
- EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE (AAAI 2024 · similarity 0.89)
- VladVA: Discriminative Fine-tuning of LVLMs (CVPR 2025 · similarity 0.89)
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.88)
- MMA: Multi-Modal Adapter for Vision-Language Models (CVPR 2024 · similarity 0.88)
- Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.