AI Paper Atlas

Research map / Large language models / Retrieval and knowledge

Are We on the Right Way to Assess Document Retrieval-Augmented Generation?

Wenxuan Shen, Mingjia Wang, Yaochen Wang, Dongping Chen, Junjie Yang, Yao Wan, Weiwei Lin

AAAI 2026 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
provide ground-truth evidence pages to MLLMs (Oracle) instead of no retrieval (ablation)Qwen2.5-VL-32B · Double-Bench (single hop)fraction fully correct: 0.200 -> 0.874Improved
provide ground-truth evidence pages to GPT-4o (Oracle) instead of no retrieval (ablation)GPT-4o · Double-Bench (single hop)fraction fully correct: 0.109 -> 0.678Improved
use a document-specific visual embedding model (ColQwen2.5-3B) instead of general multimodal embeddingsdocument retrieval · Double-Benchhit@5: 0.720 -> 0.795Improved
use visual embeddings instead of text embeddingsdocument retrieval · Double-Benchhit@5: 0.776 -> 0.795Mixed
increase number of hops in queryembedding models · Double-Benchhit@5: 0.895 -> 0.693 (colqwen, single vs 2-hop)Hurt
use a single MLLM pass with strong retriever (Colqwen-gen) instead of multi-agent pipeline (MDocAgent)document RAG · Double-Bench (average)retrieval hit@5: 0.688 -> 0.795Improved
use complex multi-agent frameworks (MDocAgent, ViDoRAG) instead of simple M3DocRAGdocument RAG · Double-Benchanswer correct given retrieval hit: 0.451 -> 0.645Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy