Research map / Vision-language and multimodal / Human and embodied modeling
MIEB: Massive Image Embedding Benchmark
Chenghao Xiao, Isaac Chung, Imene Kerboua, Jamie Stirling, Xin Zhang, Márton Kardos, Roman Solomatin, Noura Al Moubayed, Kenneth Enevoldsen, Niklas Muennighoff
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MLLM-based embedding models instead of CLIP-style models | MIEB · MIEB Full overall | Mean (all tasks): Voyage 62.5 / E5-V 58.2 vs best CLIP-style siglip 57.1 | Improved |
| MLLM-based embedding versus CLIP-style on linear probing and zero-shot classification | E5-V · MIEB Zero-shot Classification | average accuracy: E5-V 50.0 vs siglip-so400m 70.8 | Hurt |
| scale OpenCLIP model size from ViT-L (430M) to ViT-H (990M) (ablation) | OpenCLIP · Document Understanding | nDCG@5: 36.26 -> 40.41 | Improved |
| use higher-quality data (DataComp) with fewer seen samples (ablation) | CLIP ViT-L · Document Understanding | nDCG@5: 36.26 -> 38.64 | Improved |
| use text-supervised CLIP models versus vision-only self-supervised models on visual STS | Vision encoders · Visual STS | Spearman: EVA-CLIP-bigE-14-plus 71.99 vs DINO-v2 12.98, MOCO-v3 14.31 | Improved |
| use few-shot linear probing with k from 8 to 256 (ablation) | MIEB linear probing · fine- and coarse-grained classification | model ranking | No clear effect |
| use a strong multilingual text encoder (frozen BGE-M3 in VISTA, multilingual SigLIP) | Image-text embedding models · Multilingual retrieval | average score across languages | Improved |
| reduce MIEB to MIEB-lite (51 tasks) | MIEB · E5-V runtime | H100 GPU hours: 264.0 -> 46.4 | Improved |
| predict MLLM benchmark performance from vision-encoder Visual STS score | Vision encoders · 16 MLLM benchmarks | correlation: over 99% correlation on OCR tasks | Improved |
Closest papers by meaning
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.90)
- Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.88)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.88)
- MMTEB: Massive Multilingual Text Embedding Benchmark (ICLR 2025 · similarity 0.87)
- MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly (NeurIPS 2025 · similarity 0.87)
- WikiDO: A New Benchmark Evaluating Cross-Modal Retrieval for Vision-Language Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
