Research map / Vision-language and multimodal / Human and embodied modeling
UC2: Universal Cross-Lingual Cross-Modal Vision-and-Language Pre-Training
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VTLM to the pre-training objectives (ablation) | UC² · Flickr30K + MSCOCO retrieval, all-language fine-tune | Meta-Ave: 85.3 -> 86.2 | Helped |
| add VTLM to the pre-training objectives (ablation) | UC² · VG VQA Japanese | accuracy: 34.1 -> 34.2 | Helped |
| add MRTM to the pre-training objectives (ablation) | UC² · VQA v2.0 Test-Dev | accuracy: 70.93 -> 71.48 | Helped |
| add MRTM to the pre-training objectives (ablation) | UC² · VG VQA Japanese | accuracy: 33.5 -> 34.2 | Helped |
| add both VTLM and MRTM (ablation) | UC² · VQA v2.0 Test-Dev | accuracy: 69.94 -> 71.48 | Helped |
| pivot multilingual cross-modal alignment on images rather than English (ablation) | UC² · Flickr30K + MSCOCO retrieval | Meta-Ave: 84.2 -> 85.3 | Helped |
| use MRTM instead of traditional MRC (ablation) | UC² · VQA v2.0 Test-Dev | accuracy: 70.60 -> 71.45 | Helped |
| use MRTM instead of traditional MRC (ablation) | UC² · Flickr30K + MSCOCO retrieval | Meta-Ave: 85.1 -> 85.3 | No clear effect |
| include more pre-training languages while keeping training time fixed (ablation) | UC² · Flickr30K English and German retrieval | average R@K | Helped |
| pivot VTLM on Chinese instead of English (ablation) | UC² · multilingual image-text retrieval | Meta-Ave: 86.2 to 85.5 | Hurt |
| pre-train with image pivoting instead of English pivoting (ablation) | UC² · MSCOCO English retrieval | AR: 88.1 -> 87.7 | Hurt |
Closest papers by meaning
- CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer (AAAI 2024 · similarity 0.93)
- Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal Retrieval (AAAI 2025 · similarity 0.90)
- LRM-LLaVA: Overcoming the Modality Gap of Multilingual Large Language-Vision Model for Low-Resource Languages (AAAI 2025 · similarity 0.90)
- LoMo: Local Modality Substitution for Deeper Vision-Language Fusion (NeurIPS 2026 · similarity 0.89)
- Embracing Language Inclusivity and Diversity in CLIP through Continual Language Learning (AAAI 2024 · similarity 0.89)
- Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.