Research map / Vision-language and multimodal / Multimodal LLMs

RobustVisRAG: Causality-Aware Vision-Based Retrieval-Augmented Generation under Visual Degradations

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RobustVisRAG, a causality-guided dual-path VisRAG encoder (non-causal degradation token plus causal semantic path) trained with NCDM and CSA losses, and releases the DVisRAG benchmark of synthetic and real degraded document images.

Problem

Vision-based RAG models degrade when document images suffer visual distortions (blur, noise, low light, shadow), because semantic and degradation factors are entangled in pretrained VLM encoders, hurting both retrieval and generation.

Approach

A dual-path vision encoder: a unidirectional non-causal token that aggregates degradation cues, and a causal path over patch tokens that is guided by these cues via contrastive (NCDM) and clean/degraded semantic alignment (CSA) losses; only the causal semantic output is used at inference.

Key results

• Retrieval MRR@10 on real degraded subset improves from 56.47 (VisRAG-Ret) to 63.82, a gain of about 7.35 points.
• Generation top-1 accuracy on real degraded subset improves from 42.99 (VisRAG) to 55.39 for RobustVisRAG; Oracle setting reaches 69.03 vs 62.68 for VisRAG-Gen.
• End-to-end top-1 accuracy on real degraded subset rises from 42.99 to 55.39, reported as a 12.40% gain.

Surprising findings

• Adversarial robustness training (FARE) yields only limited improvement under the complex real-world degradations in DVisRAG.
• A two-stage restoration-then-VisRAG pipeline gives limited or no gains and even underperforms vanilla VisRAG on the real subset for retrieval (53.59 vs 56.47).

Limitations

• Degradation types are limited to 12 synthetic and 5 real categories captured with a single camera, so generalization to other distortions is untested.
• Real-world subset is small (1,891 test pairs) and captured under controlled conditions.
• Paper notes the non-causal and causal path design depends on the encoder architecture; generalization beyond MiniCPM-V backbones is not shown.

Experimental setup

Datasets: VisRAG dataset ArXivQA ChartQA PlotQA InfoVQA MP-DocVQA SlideVQA Synthetic (VisRAG) RVL-CDIP UniRestore degradation pipeline

Benchmarks: VisRAG dataset DVisRAG synthetic subset DVisRAG real subset

Metrics: MRR@10 Accuracy (5% relative numerical tolerance)

Baselines: BM25 BGE-large NV-Embed-v2 SigLIP ColPali VisRAG-Ret VisRAG-Ret-FM (FARE) GPT-4o MiniCPM (T) VisRAG-Gen VisRAG-Gen-FM (FFT) VisRAG-Gen-FM (PEFT)

Builds on: VisRAG FARE UniRestore MiniCPM-V

Closest papers by meaning

• VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents (ICLR 2025 · similarity 0.88)
• Revisiting the Role of Language Priors in Vision-Language Models (ICML 2024 · similarity 0.87)
• AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding (NeurIPS 2025 · similarity 0.87)
• Unified Generative and Discriminative Training for Multi-modal Large Language Models (NeurIPS 2024 · similarity 0.87)
• Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding (AAAI 2026 · similarity 0.87)
• RealRAG: Retrieval-augmented Realistic Image Generation via Self-reflective Contrastive Learning (ICML 2025 · similarity 0.87)