Research map / Vision-language and multimodal / Cross-modal learning
MuCo: Multi-turn Contrastive Learning for Multimodal Embedding Model
Geonmo Gu, Byeongho Heo, Jaemyung Yu, Jaehui Hwang, Taekyung Kim, Sangmin Lee, HeeJae Jun, Yoohoon Kang, Sangdoo Yun, Dongyoon Han
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale turns to 7 per image at fixed batch 1024 instead of scaling batch size (ablation) | MuCo-2B · MMEB zero-shot | Precision@1: 57.5 -> 58.2 with 18.0 vs 122.7 PFLOPs | Improved |
| increase single-turn batch size 1024 to 8192 (ablation) | single-turn baseline · MMEB zero-shot | Precision@1: 57.1 -> 57.8 | Improved |
| add compounded supervision (turns attend to earlier turns) (ablation) | MuCo-2B · MMEB fine-tuned | Precision@1: 68.4 -> 69.5 | Improved |
| add logit masking of same-image pairs during fine-tuning (ablation) | MuCo-2B · MMEB fine-tuned | Precision@1: 30.9 -> 69.5 | Improved |
| add logit masking in pretraining (ablation) | MuCo-2B · MMEB fine-tuned | Precision@1: 69.2 -> 69.5 | Improved |
| use multi-turn MuCo pretraining instead of single-turn on mmE5 data (ablation) | MuCo-2B · MMEB fine-tuned | Precision@1: 68.6 -> 69.0 | Improved |
| add M3T pretraining at 5M scale (ablation) | MuCo-2B · MMEB fine-tuned | Precision@1: 68.5 -> 69.5 | Improved |
| pretrain on MegaPairs 26M single-turn (ablation) | MuCo-2B · MMEB zero-shot | Precision@1: 55.6 (mmE5) -> 41.5 | Hurt |
| add each task category in data (cumulative: classification, global, local, creative VQA) (ablation) | MuCo-2B pretraining · MMEB zero-shot | Overall: 55.1 -> 58.2 | Improved |
| use 50% counterpart masking vs 25% or 75% (ablation) | MuCo-2B fine-tuning · MMEB fine-tuned | Precision@1: 69.3/68.9 -> 69.5 | Improved |
| add reconstruction guidance prompt vs none (ablation) | MuCo-2B fine-tuning · MMEB fine-tuned | Precision@1: 69.0 -> 69.5 | Improved |
| swap image captioner BLIP-Large for Qwen2-VL-7B (ablation) | MuCo-2B fine-tuning · MMEB fine-tuned | Precision@1: 69.4 vs 69.5 | No clear effect |
Closest papers by meaning
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.90)
- Scaling Language-centric Omnimodal Representation Learning (NeurIPS 2025 · similarity 0.89)
- Beyond Global Similarity: Multi-Conditional Retrieval for Fine-Grained Cross-Modal Understanding (CVPR 2026 · similarity 0.89)
- Bridging Modalities: Improving Universal Multimodal Retrieval by Multimodal Large Language Models (CVPR 2025 · similarity 0.89)
- MM-EMBED: UNIVERSAL MULTIMODAL RETRIEVAL WITH MULTIMODAL LLMS (ICLR 2025 · similarity 0.89)
- LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
