Research map / Vision-language and multimodal / CLIP and zero-shot
Efficient Discriminative Joint Encoders for Large Scale Vision-Language Reranking
Mitchell Keren Taraday, Shahaf Wagner, Chaim Baskin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add EDJE re-ranker on top of embedding model | CLIP ViT-B/16 · Flickr30K zero-shot text-to-image | R@1: 62.1 -> 76.8 | Improved |
| add EDJE re-ranker on top of embedding model | SigLIP2 ViT-L/16 384 · Flickr30K zero-shot text-to-image | R@1: 82.3 -> 87.8 | Improved |
| add EDJE re-ranker on top of embedding model | DFN ViT-L/14 · Flickr30K zero-shot text-to-image | R@1: 75.1 -> 77.5 | Improved |
| add masked language modeling to ITM (ablation) | EDJE local, SigLIP2 L 384 · Flickr30K | R@1: 82.3 -> 85.5 | Improved |
| add text-embedding recovery (ITC) to ITM+MLM (ablation) | EDJE local, SigLIP2 L 384 · Flickr30K | R@1: 85.5 -> 87.8 | Improved |
| add local-to-compressed distillation (ablation) | EDJE Compressed-64 · Flickr30K | R@1: 83.8 -> 86.9 | Improved |
| use learned attention adapter instead of striding (ablation) | EDJE 64 tokens · Flickr30K zero-shot text-to-image | R@1: 83.24 -> 86.9 | Improved |
| use learned attention adapter instead of k-means clustering (ablation) | EDJE 64 tokens · Flickr30K zero-shot text-to-image | R@1: 85.66 -> 86.9 | Improved |
| use learned attention adapter instead of attention pruning (ablation) | EDJE 64 tokens · Flickr30K zero-shot text-to-image | R@1: 82.4 -> 86.9 | Improved |
| mine negatives with CLIP L instead of SigLIP2 (ablation) | SigLIP2-based EDJE · Flickr30K zero-shot text-to-image | R@1: 87.8 -> 81.5 | Hurt |
| mine negatives with SigLIP2 L instead of CLIP (ablation) | CLIP-based EDJE · Flickr30K zero-shot text-to-image | R@1: 81.9 -> 81.8 | No clear effect |
| compress 576 vision tokens to 64 with adapter | EDJE SigLIP2 L · Flickr30K zero-shot text-to-image | R@1: 87.8 -> 86.9 | Hurt |
Closest papers by meaning
- Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs (NeurIPS 2026 · similarity 0.87)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.87)
- Attention-aware Inference Optimizations for Large Vision-Language Models with Memory-efficient Decoding (CVPR 2026 · similarity 0.87)
- DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment (CVPR 2025 · similarity 0.87)
- VladVA: Discriminative Fine-tuning of LVLMs (CVPR 2025 · similarity 0.87)
- Unified Generative and Discriminative Training for Multi-modal Large Language Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
