Research map / Vision-language and multimodal / Cross-modal learning

BayesRAG: Probabilistic Mutual Evidence Corroboration for Multimodal Retrieval-Augmented Generation

NeurIPS 2026 · Vision-language and multimodal › Cross-modal learning · approach: Probabilistic embeddings for cross-modal retrieval · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BayesRAG, a multimodal RAG retrieval method that fuses text, image, and screenshot candidates via Bayesian posterior scoring with Dempster-Shafer likelihoods and graph or layout consistency priors, reporting gains on DocBench and MMLongBench-Doc.

Problem

Multimodal RAG over visually rich documents ranks text, image, and page candidates independently by cosine similarity, so high-scoring but semantically unrelated or spatially disconnected cross-modal evidence gets retrieved together.

Approach

Treat retrieval as Bayesian evidence fusion: likelihood from Dempster-Shafer combination of normalized per-modality similarity masses, multiplied by a consistency prior from either a knowledge-graph edge-connectivity score or a bounding-box layout proximity check, so that mutually corroborating text-image-screenshot tuples rank higher.

Key results

• On MMLongBench-Doc with GPT-4o-mini generator, BayesRAG reaches 38.8% accuracy and 44.1% LLM score, versus 37.1% and 41.5% for RAGAnything.
• On DocBench, BayesRAG reaches 51.2% overall accuracy, vs. 48.7% for RAGAnything (GPT-4o-mini generator).
• Recall@20 on MMLongBench-Doc rises from 56.6% (vector retrieval) to 76.6% (BayesRAG); with Qwen3-VL-32B as generator, MMLongBench-Doc score reaches 59.4%.

Surprising findings

• Retrieval recall of 76.6% at Top-20 does not translate to proportional generation gains; the authors attribute much of the error to query ambiguity in MMLongBench-Doc (51% of 100 sampled failures).
• Feeding 50 page screenshots of the target document directly to GPT-4o-mini outperforms several RAG pipelines, per the authors' analysis.

Limitations

• Retrieval-generation gap: high recall is bounded by the downstream MLLM's ability to use visual evidence.
• Advantages are less pronounced for text-only or large-table documents such as some financial reports.
• Relies on a knowledge graph built with Qwen3-VL-32B, and the Layout Prior uses hand-set thresholds and penalties; the hyperparameters alpha, beta, and kappa are fixed empirically without sensitivity analysis reported.

Experimental setup

Datasets: DocBench MMLongBench-Doc

Benchmarks: DocBench MMLongBench-Doc

Metrics: accuracy LLM-based score Recall@K

Baselines: GPT-4o-mini VisRAG ViDoRAG RAGFlow RAGAnything Embedding-RAG

Builds on: Dempster-Shafer theory VisRAG ViDoRAG RAG-Anything ColPali

Closest papers by meaning

• ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge (NeurIPS 2026 · similarity 0.89)
• Benchmarking Retrieval-Augmented Multimomal Generation for Document Question Answering (NeurIPS 2025 · similarity 0.89)
• Beyond Global Similarity: Multi-Conditional Retrieval for Fine-Grained Cross-Modal Understanding (CVPR 2026 · similarity 0.88)
• TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning (NeurIPS 2026 · similarity 0.88)
• RMIR: A Benchmark Dataset for Reasoning-Intensive Multimodal Image Retrieval (CVPR 2026 · similarity 0.88)
• MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval (ICLR 2026 · similarity 0.88)