Research map / Vision-language and multimodal / Multimodal LLMs
Suit the Remedy to the Retriever: Interpretable Query Optimization with Retriever Preference Alignment for Vision-Language Retrieval
GuangHao Meng, Jinpeng Wang, Jieming Zhu, Letian Zhang, Yong Jiang, Dan Zhao, Qing Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes interpretable query optimization that aligns query rewriting with each retriever's preferences for vision-language retrieval (abstract only available).
Problem
Challenging concepts in text or image queries confuse vision-language retrievers, and existing query optimization methods ignore what text descriptions retrievers prefer.
Approach
Optimizes queries interpretably while aligning them to retriever preferences, i.e., rewriting queries into descriptions that better match a given retriever's embedding space. Details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add QRE-SFT on preference data to ReCoT rewriter (ablation) | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | I2T R@1: 74.4 -> 75.5 | Improved |
| add CCD-SFT (DCCR) on top of QRE-SFT (ablation) | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | I2T R@1: 75.5 -> 76.0 | Improved |
| add preference rank optimization (DPRO) (ablation) | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | I2T R@1: 76.0 -> 77.0 | Improved |
| add preference rank optimization (DPRO) (ablation) | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | T2I R@1: 54.0 -> 55.1 | Improved |
| add ReCoT detect-then-rewrite (ablation) | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | I2T R@1: 76.1 -> 77.0 | Improved |
| add ReCoT without preference data (ablation) | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | I2T R@1: 74.2 -> 74.4 | No clear effect |
| full preference post-training vs ReCoT only (ablation) | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | I2T R@1: 74.4 -> 77.0 | Improved |
| RAQO vs LLMsRewrite (retriever-agnostic rewriting) | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | I2T R@1: 74.2 -> 77.0 | Improved |
| apply RAQO to BLIP2 retriever | RAQO (CLIP ViT-B/32 retriever) · Flickr30k-CFQ | I2T R@1: 85.2 -> 86.0 | Improved |
| apply RAQO to CLIP, zero-shot to WikiDO OOD | RAQO (CLIP ViT-B/32 retriever) · WikiDO OOD | I2T R@1: 73.4 -> 73.9 | Improved |
| apply RAQO on Fashion200K | RAQO (CLIP ViT-B/32 retriever) · Fashion200K | I2T R@1: 9.5 -> 10.9 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- RACLIP — RA-CLIP: Retrieval Augmented Contrastive Language-Image Pre-Training (CVPR 2023) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Limitations
- Only the abstract was available; full method and results could not be verified.
Closest papers by meaning
- SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledge (NeurIPS 2024 · similarity 0.89)
- Revisiting the Role of Language Priors in Vision-Language Models (ICML 2024 · similarity 0.89)
- Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal Retrieval (AAAI 2025 · similarity 0.89)
- Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models (ICML 2024 · similarity 0.89)
- WikiDO: A New Benchmark Evaluating Cross-Modal Retrieval for Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- Beyond Global Similarity: Multi-Conditional Retrieval for Fine-Grained Cross-Modal Understanding (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
