Research map / Vision-language and multimodal / Human and embodied modeling
Positional Bias in Multimodal Embedding Models: Do They Favor the Beginning, the Middle, or the End?
Kebin Wu, Fatima Albreiki
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on shuffled sub-captions instead of original caption order (ablation) | Long-CLIP (ViT-B/16) · Urban1K text encoder (T2I) | Recall@1 drop when first segment moves start to end: drop 0.303 (0.581 -> 0.278) vs 0.199 (0.581 -> 0.382) | Improved |
| use relative positional encoding (RoPE, TULIP) instead of absolute encoding (Long-CLIP) (ablation) | CLIP-style text encoder · Urban1K text encoder (T2I) | Recall@1 when first segment moves from position 0 to 2: TULIP 0.66 -> 0.065 vs Long-CLIP 0.582 -> 0.485 | Hurt |
| use sigmoid loss (SigLIP) instead of softmax contrastive loss (ablation) | CLIP-style models · Image-text retrieval positional bias | positional bias pattern | No clear effect |
| scale model from ViT-B/16 to ViT-L/14 (ablation) | CLIP · Urban1K/COCO positional bias | coefficient of variation across positions | Hurt |
| increase image resolution from 224 to 336 (ViT-L/14) (ablation) | CLIP image encoder · Urban1K image positional bias | coefficient of variation across positions: lower in 5 of 7 cases | Improved |
| use smaller patch size (B/16 instead of B/32) (ablation) | CLIP image encoder · Urban1K image positional bias | coefficient of variation across positions | Improved |
| use ResNet-50 CNN image encoder instead of ViT (ablation) | CLIP · Urban1K/ImageNet positional bias | positional bias presence | No clear effect |
| train vision encoder with image-text pairs (CLIP) instead of image-only (ablation) | ResNet-50 / ViT-B/16 image encoder · ImageNet classification with shifted segment | top-1 accuracy variation across positions | Hurt |
| short captions (COCO) instead of long captions (ablation) | CLIP-ViT-B/16 text encoder · COCO | Recall@10 positional bias | No clear effect |
Closest papers by meaning
- On the Value of Cross-Modal Misalignment in Multimodal Representation Learning (NeurIPS 2025 · similarity 0.88)
- Eliminating Position Bias of Language Models: A Mechanistic Approach (ICLR 2025 · similarity 0.88)
- CLIP the Bias: How Useful is Balancing Data in Multimodal Learning? (ICLR 2024 · similarity 0.88)
- Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs (NeurIPS 2024 · similarity 0.87)
- Revisiting Multimodal Positional Encoding in Vision–Language Models (ICLR 2026 · similarity 0.87)
- Understanding the Emergence of Multimodal Representation Alignment (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
