Research map / Vision-language and multimodal / Multimodal LLMs
VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VRAG-RL, an RL framework (GRPO) that trains Qwen2.5-VL VLMs to iteratively search visually rich document collections and crop/zoom regions, with a retrieval-NDCG plus model-judged answer reward.
Problem
Traditional RAG handles visually rich documents poorly; existing vision-based RAG uses fixed pipelines, under-activates VLM reasoning, and issues poor search queries that fail to retrieve relevant images.
Approach
Combine a visual perception action space (region selection, cropping, re-encoding at higher resolution for coarse-to-fine viewing) with multi-turn search and a reward mixing retrieval NDCG, model-based answer correctness, and a pattern reward, optimized with GRPO after SFT on multi-expert trajectories.
Key results
- VRAG-RL on Qwen2.5-VL-7B reaches 57.1 overall accuracy vs 37.4 for Search-R1-VL.
- VRAG-RL on Qwen2.5-VL-3B reaches 53.5 overall vs 21.3 for Search-R1-VL.
- RL lowers invalid action rate from 9.4% (SFT) to 5.1% and raises finish rate from 84.2% to 97.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VRAG-RL instead of Search-R1-VL | Qwen2.5-VL-3B-Instruct · Overall | accuracy: 21.3 -> 53.5 | Helped |
| Use VRAG-RL instead of Search-R1-VL | Qwen2.5-VL-7B-Instruct · Overall | accuracy: 37.4 -> 57.1 | Helped |
| Add visual-perception actions to vanilla reward and search actions (ablation) | VRAG-RL · Three benchmarks | accuracy: 47.2 -> 49.3 | Helped |
| Add visual-perception actions to RAG-specific reward and search actions (ablation) | VRAG-RL · Three benchmarks | accuracy: 54.9 -> 57.1 | Helped |
| Replace vanilla reward with RAG-specific reward while retaining search actions (ablation) | VRAG-RL · Three benchmarks | accuracy: 47.2 -> 54.9 | Helped |
| Add RL to SFT | VRAG-RL · Multi-step interactions | Invalid Action Rate: 9.4 -> 5.1 | Helped |
| Add RL to SFT | VRAG-RL · Multi-step interactions | Finish Rate: 84.2 -> 97.1 | Helped |
Surprising findings
- Ablation shows the RAG-specific retrieval reward contributes more gain (47.2 to 54.9) than the visual perception action space alone (47.2 to 49.3) on Qwen2.5-VL-7B.
- Authors note that recall/EM rewards can be hacked, causing repetitive responses and unstable training.
Limitations
- Multi-turn interaction increases latency relative to single-pass vanilla RAG.
- Relies on a 7B model-based reward for correctness judgments, and the authors aim to reduce hallucinations via stronger models.
- Action space limited to search, region, and answer; more human-like actions left to future work.
Experimental setup
Datasets: SlideVQA ViDoSeek MMLongBench
Benchmarks: SlideVQA ViDoSeek MMLongBench
Metrics: accuracy (model-judged binary)
Baselines: Vanilla RAG ReAct Search-R1 Search-R1-VL
Builds on: GRPO Search-R1 ReAct ColPali
Closest papers by meaning
- SenseSearch: Empowering Vision-Language Models with High-Resolution Agentic Search-Reasoning via Reinforcement Learning (CVPR 2026 · similarity 0.90)
- Fix Before Search: Benchmarking Agentic Visual Query Pre-processing in Multimodal Retrieval-augmented Generation (ICML 2026 · similarity 0.89)
- Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graph (ICML 2026 · similarity 0.88)
- NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation (NeurIPS 2025 · similarity 0.88)
- Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.88)
- Grounded Reinforcement Learning for Visual Reasoning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.