Research map / Vision-language and multimodal / Multimodal LLMs

Illuminating Visual Identity in Universal Multimodal Embeddings

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Defines visual identity discrimination for universal multimodal embeddings, builds the MVEB benchmark (28 sub-datasets, ~522K samples), and trains VisME with identity-aware batch sampling and hard-negative mining on Qwen2-VL/Qwen2.5-VL via LoRA.

Problem

Existing universal multimodal embedding models fail to discriminate visual identities (instances, people, products) conditioned on text instructions, since benchmarks like MMEB include almost no identity-level tasks.

Approach

Formalize visual identity discrimination as four meta-tasks (identity recognition, re-ID, identity grounding, identity editing), curate a benchmark for them, and train UMEs with identity-aware offline batch sampling so that same-identity samples are not used as false negatives, plus structured hard-negative sampling, under a unified contrastive loss.

Key results

• VisME (Qwen2.5-VL-7B) reaches 78.8 average on MVEB versus 52.7 for B3 (7B), and 72.2 on MMEB average versus 72.0 for B3.
• VisME (Qwen2.5-VL-3B) achieves 69.6 MMEB average and 76.7 MVEB average, outperforming the 2B baselines.
• ID-aware sampling raises MVEB in-distribution accuracy from 64.3 to 76.2 in the ablation.

Surprising findings

• LamRA (Qwen2.5-VL-7B) scores only 20.2 on MVEB despite 52.4 on MMEB, showing identity discrimination is not captured by general benchmarks.
• Single-benchmark training (MMEB only or MVEB only) fails to generalize to the other benchmark.

Limitations

• MVEB candidate pools are about 3k per sub-dataset; performance drops notably when scaling to 100k candidates.
• Some MVEB sub-datasets have original split overlaps, and identity-aware split is not always feasible (random sample-level split fallback).

Experimental setup

Datasets: MMEB MVEB GLDv2 GPTImageEdit iNat SOP Cars196

Benchmarks: MMEB MVEB

Metrics: Precision@1

Baselines: CLIP SigLIP2 LLM2CLIP VLM2Vec GME UniME LLaVE VLM2Vec-V2.0 B3 UniME-v2 LamRA

Builds on: VLM2Vec MMEB GradCache LoRA

Closest papers by meaning

• VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.88)
• UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings (ICLR 2026 · similarity 0.87)
• DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception (NeurIPS 2024 · similarity 0.86)
• ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation (NeurIPS 2026 · similarity 0.86)
• UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.86)
• Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillation (NeurIPS 2025 · similarity 0.86)