Research map / Vision-language and multimodal / Cross-modal learning
Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibility
Jaeseok Byun, Gukyeong Kwon, Han-Kai Hsu, MEHER GITIKA KARUMURI, Zhikang Zhang, Hao Yang, Davide Modolo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-level preservation loss plus focal re-weighting to the base adapter | L2 base adapter · MMEB, CLIP->UniME-7B | Precision@1: 20.03 -> 49.56 | Improved |
| add MKP-Adapter | base adapter · COCO/Flickr30K/Urban1K, CLIP->UniME | I2T Recall@1: 63.35 -> 77.76 | Improved |
| use L2-based adapters | base/EC-style · T2I retrieval | Recall@1: E5-V old 72.41 vs Base 72.11 | Hurt |
| use contrastive base adapter | CL-based adapter · I2T retrieval, CLIP->GME | Recall@1: 69.80 -> 64.59 | Hurt |
| add distance-wise preservation loss (ablation) | MKP-Adapter · I2T avg, CLIP->UniME | Recall@1: 71.74 -> 77.76 | Improved |
| add angle-wise preservation loss (ablation) | MKP-Adapter · MMEB, CLIP->UniME | Precision@1: 41.55 -> 49.56 | Improved |
| add point-wise preservation loss (ablation) | MKP-Adapter · MMEB, CLIP->UniME | Precision@1: 47.43 -> 49.56 | Improved |
| add point-wise preservation loss (ablation) | MKP-Adapter · T2I avg, CLIP->UniME | Recall@1: 63.22 (w/o) vs 63.07 | No clear effect |
| add focal re-weighting (ablation) | MKP-Adapter · MMEB, CLIP->UniME | Precision@1: 46.61 -> 49.56 | Improved |
| use a single shared adapter instead of separate per-modality adapters (ablation) | MKP-Adapter · I2T/T2I, E5-V->UniME | Recall@1: base L2 72.93 vs separate L2 72.30 | Improved |
| use L2 instead of L1 base loss (ablation) | base adapter · I2T avg, E5-V->UniME | Recall@1: 72.03 -> 72.93 | Improved |
| increase adapter size or training data from 558k to 4M pairs (ablation) | MKP-Adapter · I2T/T2I, E5-V->UniME | Recall@1: 82.03 -> 82.54 (4M pairs) | No clear effect |
Closest papers by meaning
- Forward Compatible Training for Large-Scale Embedding Retrieval Systems (CVPR 2022 · similarity 0.88)
- Adaptive Prompt-Based Semantic Embedding with Inspire Potential of Implicit Knowledge for Cross-Modal Retrieval (AAAI 2025 · similarity 0.85)
- FreeRet: MLLMs as Training-Free Retrievers (ICML 2026 · similarity 0.85)
- NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models (ICLR 2025 · similarity 0.85)
- MoDA: Modulation Adapter for Fine-Grained Visual Understanding in Instructional MLLMs (ICML 2026 · similarity 0.84)
- Online Adaptation of Language Models with a Memory of Amortized Contexts (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
