Research map / Vision-language and multimodal / Multimodal LLMs

Think Then Embed: Generative Context Improves Multimodal Embedding

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Think-Then-Embed (TTE), which generates embedding-centric reasoning traces with an MLLM reasoner before embedding, achieving state-of-the-art on MMEB-V1 and MMEB-V2 with 2B/7B Qwen2-VL embedders, and explores distilled and unified reasoner-embedder variants.

Problem

Universal multimodal embedding (UME) models that encode query and instruction directly struggle with complex instructions requiring reasoning; MLLMs are used only as encoders, leaving their generative capacity unused.

Approach

Insert an explicit thinking stage: a reasoner MLLM generates embedding-centric reasoning (ECR) traces conditioned on the instruction, and the embedder conditions on the original input plus this trace; variants include a 72B teacher reasoner, an SFT-distilled same-size reasoner, and a unified reasoner with a pluggable embedding head.

Key results

• TTE-7B with teacher reasoner (TTE_t) reaches 71.5 overall on MMEB-V2, the top of the leaderboard at submission, surpassing seed-1.6-embedding (71.3).
• On MMEB-V1, TTE_t-7B reaches 78.5 overall versus 72.5 for the next best model (QQMM), and TTE_s-2B reaches 68.7 overall, the best among ~2B models.
• Two-stage unified reasoner-embedder (TTE_u-2B) matches the separate SFT-reasoner variant (68.8 vs 68.7 on MMEB-V1 overall) while using a single model.

Surprising findings

• Joint SFT plus contrastive training of one backbone degraded performance relative to the baseline embedder, which the authors attribute to conflicting gradients.
• Training the embedder with noisy or imperfect ECRs gave a 1-2 point gain, and ECR text alone with an off-the-shelf text encoder performed much worse than the embedder.

Limitations

• Teacher reasoner (72B) is expensive to run for ECR generation.
• Reasoning traces are not directly optimized for retrieval signal; RL-based ECR optimization left to future work.
• Using a different reasoner can change embedder performance, so the method depends on reasoner quality.

Experimental setup

Datasets: MMEB-V1 training data MMEB-V2 training data

Benchmarks: MMEB-V1 MMEB-V2 Flickr30k MSCOCO

Metrics: Precision@1 NDCG@5

Baselines: CLIP SigLIP UniIR MagicLens VLM2Vec-V1 VLM2Vec-V2 UniME LLaVE B3 GME LamRA seed-1.6-embedding

Builds on: VLM2Vec MMEB chain-of-thought prompting InfoNCE LoRA

Closest papers by meaning

• UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings (ICLR 2026 · similarity 0.93)
• Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.89)
• Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration (ICML 2025 · similarity 0.88)
• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.88)
• VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.88)
• MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.88)