Research map / Vision-language and multimodal / CLIP and zero-shot
FAM: Fine-Grained Alignment Matters in Multimodal Embedding Learning with Large Vision-Language Models
Tianhang Xiang, Yirui Li, Lizhao Liu, Hongyan Zhi, Chuanshen Chen, Qing Du, Mingkui Tan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MAC to VLM2Vec (ablation) | VLM2Vec Qwen2-VL-2B · MMEB (Qwen2-VL-2B) | average score: 55.0 -> 56.4 | Improved |
| add VEIN to VLM2Vec (ablation) | VLM2Vec Qwen2-VL-2B · MMEB (Qwen2-VL-2B) | average score: 55.0 -> 55.9 | Improved |
| add VEIN on top of MAC (ablation) | FAM · MMEB (Qwen2-VL-2B) | average score: 56.4 -> 57.3 | Improved |
| add MAC on top of VEIN (ablation) | FAM · MMEB (Qwen2-VL-2B) | average score: 55.9 -> 57.3 | Improved |
| add extra alignment image-text data as retrieval data instead of MAC (ablation) | VLM2Vec · MMEB (Qwen2-VL-2B) | average score: 55.0 -> 55.3 | No clear effect |
| use MAC training strategy rather than plain extra data (ablation) | VLM2Vec · MMEB (Qwen2-VL-2B) | average score: 55.3 -> 56.4 | Improved |
| add coarse-to-fine loss Lc2f to Lc (ablation) | MAC alignment stage · MMEB (Qwen2-VL-2B) alignment only | average score: 34.0 -> 35.3 | Improved |
| add fine-grained loss Lf (ablation) | MAC alignment stage · MMEB (Qwen2-VL-2B) alignment only | average score: 35.3 -> 35.4 | Improved |
| add Lc2f to Lc (ablation) | MAC, full pipeline · MMEB (Qwen2-VL-2B) align + adapt | average score: 55.8 -> 56.1 | Improved |
| add Lf to Lc and Lc2f (ablation) | MAC, full pipeline · MMEB (Qwen2-VL-2B) align + adapt | average score: 56.1 -> 56.4 | Improved |
| apply VEIN at all layers instead of none (ablation) | VLM2Vec · MMEB (Qwen2-VL-2B) | average score: 55.0 -> 54.2 | Hurt |
| apply VEIN at last three layers instead of last layer only (ablation) | VEIN · MMEB (Qwen2-VL-2B) | average score: 55.3 -> 55.9 | Improved |
Closest papers by meaning
- Improving fine-grained understanding in image-text pre-training (ICML 2024 · similarity 0.92)
- Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models (NeurIPS 2025 · similarity 0.92)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.91)
- Scaling Language-centric Omnimodal Representation Learning (NeurIPS 2025 · similarity 0.90)
- GranViT: A Fine-Grained Vision Model With Autoregressive Perception For MLLMs (ICLR 2026 · similarity 0.90)
- Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Models (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
