AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents

Dannong Xu, zhongyu yang, Jun Chen, Yingfang Yuan, Ming Hu, Lei Sun, Luc Van Gool, Danda Paudel, Chun-Mei Feng

ECCV 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
move from single-modality to cross-modality candidate pool (ablation)E5-V · MultiHaystack retrieval (overall)Recall@1 (%): 50.87 -> 40.83Hurt
move from single-modality to cross-modality candidate pool (ablation)SigLIP2 · MultiHaystack retrieval (overall)Recall@1 (%): 51.81 -> 40.96Hurt
move from single-modality to cross-modality candidate pool (ablation)CLIP · MultiHaystack retrieval (overall)Recall@1 (%): 36.28 -> 26.10Hurt
enlarge candidate pool from 1K to 46K (ablation)E5-V · MultiHaystack retrievalRecall@1 (%): 72.42 -> 40.83Hurt
provide gold evidence versus top-5 E5-V retrieval (ablation)GPT-5 · MultiHaystack reasoningaccuracy (%): 80.86 -> 51.4Hurt
retrieve cross-modality instead of single-modality (ablation)GPT-5 · MultiHaystack reasoning (E5-V top-5)accuracy (%): 59.84 -> 51.41Hurt
retrieve cross-modality instead of single-modality (ablation)Gemini-2.5-Flash · MultiHaystack reasoning (E5-V top-5)accuracy (%): 50.87 -> 43.64Hurt
add query refinement to E5-V (ablation)E5-V · MultiHaystack retrieval (cross-modality)Recall@1 (%): 40.83 -> 43.78Improved
add MMSearch pipeline to E5-V (ablation)E5-V · MultiHaystack retrieval (cross-modality)Recall@1 (%): 40.83 -> 44.58Improved
use VisRAG instead of E5-VE5-V · MultiHaystack retrieval (cross-modality)Recall@1 (%): 40.83 -> 45.38Mixed
increase retrieved items from top-1 to top-5 (ablation)GPT-5 with E5-V · MultiHaystack reasoningaccuracy (%): 64.11 at Top-5Improved
mix modalities in the pool at same total size versus single-modality pool (ablation)MM-Embed · MultiHaystack retrievalRecallHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy