AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

VL-DocIR: A Benchmark for Vision-Based Long Document Retrieval

Simon Ott, Yufan Chen, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VL-DocIR, a page-level benchmark for evidence-complete vision-based long document retrieval where answers may span multiple pages or documents.

Problem

Current vision-based document retrieval evaluations emphasize short documents and single relevant pages, missing realistic long-document retrieval where evidence may span multiple pages or documents.

Approach

A page-level benchmark targeting evidence-complete retrieval over long visual documents, requiring all evidence pages needed to answer a query to be retrieved rather than a single relevant page.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use multi-vector instead of single-vector retrievalJina V4 · VL-DocIR MacronDCGAll@10: 48.03 -> 59.59Improved
use multi-vector instead of single-vector retrievalJina V4 · VL-DocIR MacroRecallAll@10: 60.24 -> 70.77Improved
add document-context scoring (ablation)ColPali v1.3 · VL-DocIR MacroRecallAll@10: 49.13 -> 65.69Improved
add document-context scoring (ablation)ColQwen2.5 · VL-DocIR MacroRecallAll@10: 67.23 -> 75.78Improved
add reranking with Jina Reranker M0 (ablation)ColNomic · VL-DocIR MacroRecallAll@10: 71.76 -> 74.62; nDCGAll@10 61.82 -> 65.00Mixed
add query splitting (ablation)ColQwen2.5 · VL-DocIR MacroRecallAll@10: 67.23 -> 69.19Improved
fuse ColNomic (Ctx) with OCR-BM25 via RRF (ablation)ColNomic (Ctx) · VL-DocIR MacronDCGAll@10 / RecallAll@10: RecallAll 78.19 -> 78.67; nDCGAll 66.03 -> 65.13Mixed
ensemble base, Ctx and rerank lists (ablation)ColNomic · VL-DocIR MacronDCGAll@10: 66.03 -> 69.28Improved
move from within-document to cross-document evidenceColNomic B+Ctx+Rank · evidence structureRecallAll@5: 66.3 -> 27.6 (Doc-Far -> Cross-Doc)Hurt
apply document-context scoring on Micro nDCGAll for ColNomic (ablation)ColNomic · VL-DocIR MicronDCGAll@10: 70.68 -> 70.07No clear effect

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Experimental setup

Benchmarks: VL-DocIR

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy