Research map / Vision-language and multimodal / Cross-modal learning
FLAVA: A Foundational Language and Vision Alignment Model
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MMM and ITM to global contrastive pretraining (ablation) | FLAVA · Multimodal tasks (average) | average score: 66.25 -> 69.11; +2.86% | Helped |
| Add MMM and ITM to global contrastive pretraining (ablation) | FLAVA · GLUE (8-task average) | average score: 64.80 -> 74.22; +9% | Helped |
| Add unimodal image and text data with MIM and MLM during joint training (ablation) | FLAVA_MM · GLUE (8-task average) | average score: 74.22 -> 75.55 | Helped |
| Add unimodal image and text data with MIM and MLM during joint training (ablation) | FLAVA_MM · All 35 tasks (macro average) | macro average: 74.23 -> 73.72 | Hurt |
| Initialize image and text encoders from unimodal self-supervised pretraining (ablation) | FLAVA without unimodal initialization · All 35 tasks (macro average) | macro average: 73.72 -> 75.85 | Helped |
| Use global rather than local contrastive backpropagation across GPUs (ablation) | FLAVA_C · All 35 tasks (macro average) | macro average: +1.65% | Helped |
| Use DINO rather than BEiT vision-encoder initialization (ablation) | FLAVA · Multimodal tasks | average score: 67.32 -> 69.92 | Helped |
| Use DINO rather than BEiT vision-encoder initialization (ablation) | FLAVA · GLUE | average score: 77.40 -> 78.19 | Mixed |
| Use FLAVA encoder architecture and tokenizer optimizations rather than the original CLIP architecture (ablation) | CLIP (PMD) · All 35 tasks (macro average) | macro average: 61.52 -> 68.51; +6.99 | Helped |
| Use global rather than local contrastive loss (ablation) | FLAVA_C · Vision tasks | average score: 76.12 -> 79.14 | Helped |
| Use global rather than local contrastive loss (ablation) | FLAVA_C · Multimodal tasks | average score: 66.42 -> 66.25 | Helped |
Closest papers by meaning
- From Pixels to Words -- Towards Native Vision-Language Primitives at Scale (ICLR 2026 · similarity 0.90)
- Vision-Language Models Create Cross-Modal Task Representations (ICML 2025 · similarity 0.89)
- Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment (ICLR 2025 · similarity 0.89)
- Post-pre-training for Modality Alignment in Vision-Language Foundation Models (CVPR 2025 · similarity 0.89)
- Assessing and Learning Alignment of Unimodal Vision and Language Models (CVPR 2025 · similarity 0.89)
- LVLM-Aided Alignment of Task-Specific Vision Models (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.