Research map / Vision-language and multimodal / Human and embodied modeling
Chatting Makes Perfect: Chat-based Image Retrieval
Matan Levy, Rami Ben-Ari, Nir Darshan, Dani Lischinski
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train retriever on dialogues instead of using off-the-shelf BLIP on caption | ChatIR · COCO | retrieval accuracy (single-hop): 83\% to 87\% | Improved |
| add dialogue rounds to the query for fine-tuned BLIP baseline | BLIP / CLIP · Flickr30K and COCO | retrieval accuracy | Improved |
| ChatIR dialogue-trained model vs BLIP fine-tuned single-hop at 0 rounds | ChatIR · VisDial val | retrieval performance: 63.66\% vs. 63.61\% | No clear effect |
| add 2 dialogue rounds with ChatGPT questioner | ChatIR · VisDial val, 50K images | Hit rate@10: 63% to 73.5% | Improved |
| ChatGPT questioner with full chat history instead of Human questions | ChatIR · VisDial val | Hit rate@10: ~2% | Improved |
| ChatGPT questioner without access to answers (Unanswered) vs Human (ablation) | ChatIR · VisDial val | Hit rate@10: ~0.5% | No clear effect |
| use RL-based questioners or FLAN models instead of ChatGPT (ablation) | ChatIR · VisDial val | Hit rate@10 / repeated questions: 1.85-3.44 | Hurt |
| replace BLIP2 answerer with Human answerer (ablation) | ChatIR with ChatGPT questioner · VisDial val subset | Hit rate@10: 81% | Improved |
| mask image captions during retriever training (ablation) | Image Retriever F · VisDial | retrieval performance: 2-3% | Improved |
| mask questions, answers, Q&A rounds or tokens instead of captions (ablation) | Image Retriever F · VisDial | retrieval performance | No clear effect |
| augment training with a second synthetic dialogue per image (ablation) | Image Retriever F · VisDial | Hits@10 and ATR | Mixed |
| add dialogues on 73K new COCO images (68% more images) (ablation) | Image Retriever F · VisDial | ATR: 4\% drop in ATR | Hurt |
Closest papers by meaning
- Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment (CVPR 2025 · similarity 0.88)
- MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions (ICML 2024 · similarity 0.86)
- Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxy (CVPR 2025 · similarity 0.86)
- F2RVLM: Boosting Fine-grained Fragment Retrieval for Multi-Modal Long-form Dialogue with Vision Language Model (AAAI 2026 · similarity 0.86)
- INQUIRE: A Natural World Text-to-Image Retrieval Benchmark (NeurIPS 2024 · similarity 0.86)
- GenIR: Generative Visual Feedback for Mental Image Retrieval (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
