Research map / Vision-language and multimodal / Multimodal LLMs

TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces TopoGraphRAG-Bench, a layout-grounded benchmark of 2,024 bottom-up multimodal questions over 201 MMDocIR documents (single-hop, bridge-chain, synthesis), and evaluates text-only GraphRAG, VisRAG, and multimodal GraphRAG systems with topology-aware metrics.

Problem

Existing GraphRAG evaluations are text-centred, and multimodal document RAG benchmarks do not diagnose whether systems recover the intended evidence topology across text, tables, figures and layouts.

Approach

Construct questions bottom-up from layout-grounded evidence units linked by shared entities into controlled reasoning topologies, validated by counterfactual checks (shortcut resistance, modality necessity, evidence necessity), and score retrieval and generation with topology-enriched references and step coverage.

Key results

• MegaRAG achieves the highest answer accuracy (0.535) and step coverage (0.733) overall, while RAG-Anything has the best context precision (0.809) and recall (0.722).
• HippoRAG drops from 0.90 accuracy on single-hop text questions to 0.17 on mixed/visual bridge-chain questions.
• VisRAG reaches only 0.26 on Bridge-MM and 0.25 on Synthesis-MM despite strong single-hop visual accuracy.

Surprising findings

• Text-only GraphRAG performs strongly on text-only questions but degrades sharply on multimodal bridge-chain questions.
• Page-level visual retrieval (VisRAG) performs competitively on single-hop visual questions but collapses on multi-hop and synthesis questions.

Limitations

• Benchmark built on a random sample of 201 MMDocIR documents and relies on LLM/VLM-generated entities and questions.
• Judge-based metrics (answer accuracy, step coverage) depend on LLM judges, with only a 100-unit human spot-check.
• VisRAG is excluded from text-based retrieval and faithfulness metrics, limiting direct comparison.

Experimental setup

Datasets: MMDocIR UniDoc-Bench

Benchmarks: TopoGraphRAG-Bench

Metrics: context precision context recall answer accuracy faithfulness response relevancy step coverage

Baselines: LightRAG HippoRAG Microsoft GraphRAG VisRAG RAG-Anything MegaRAG

Builds on: MMDocIR MuSiQue RAGAS VisRAG

Closest papers by meaning

• Benchmarking Retrieval-Augmented Multimomal Generation for Document Question Answering (NeurIPS 2025 · similarity 0.90)
• ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge (NeurIPS 2026 · similarity 0.89)
• DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents (ICML 2026 · similarity 0.88)
• BayesRAG: Probabilistic Mutual Evidence Corroboration for Multimodal Retrieval-Augmented Generation (NeurIPS 2026 · similarity 0.88)
• DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding (NeurIPS 2026 · similarity 0.88)
• Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning (AAAI 2026 · similarity 0.88)