Research map / Large language models / Retrieval and knowledge

AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generation

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AgenticOCR, a query-driven agentic parser built on Qwen3-VL that zooms, rotates and OCRs only query-relevant page regions, trained by SFT on Gemini-distilled trajectories then GRPO, and plugged into visual RAG pipelines.

Problem

Page-level retrieval in visual RAG delivers entire pages with extraneous content, diluting the generator's attention and exceeding visual token budgets, which causes detail loss and hallucination.

Approach

Reframe OCR as an on-demand, instruction-following agentic process: a VLM invokes a unified zoom/rotate/OCR tool to extract only query-relevant evidence regions from retrieved pages, trained with dual-recall rewards and penalties for spurious, redundant, or oversized boxes.

Key results

• On MMLongBench-Doc, AgenticOCR-8B with Evidence+OCR input reaches 66.4 overall accuracy, versus 67.6 for DocLens with Gemini-2.5-Pro and 65.8 for human experts.
• On FinRAGBench-V, AgenticOCR-8B with Evidence+OCR reaches 78.6 accuracy, above SimpleDoc and DocLens (best 70.4) reported in the table.
• AgenticOCR-4B reaches 97.3 Page_acc and 83.7 Recall_min on the FinRAGBench-V bounding-box subset, comparable to Gemini-3-Pro-Preview.

Surprising findings

• Evidence-based inputs increased Gemini-2.5-Pro token consumption on MMLongBench-Doc due to its fixed per-image token allocation, exceeding the page+OCR baseline.
• Adding model-generated comments sometimes introduced hallucinations, making Evidence+OCR only marginally better than Evidence on several subsets.

Limitations

• Lower retrieval precision (28.0% MMLongBench-Doc, 34.8% FinRAGBench-V) and weak unanswerable-question accuracy (66.8 vs DocLens up to 74.0).
• Weak on table extraction, often returning incomplete cells or rows without headers.
• Single-page processing limits cross-page reasoning and the model can select wrong regions on hard distractors.

Experimental setup

Datasets: ViDoRe-v3 CodeVision rotated/complex layout samples MMLongBench-Doc FinRAGBench-V MVToolbench

Benchmarks: MMLongBench-Doc FinRAGBench-V MVToolbench

Metrics: accuracy Page_acc Recall_min Recall_EM Precision_min F1_min input tokens output tokens

Baselines: GPT-4o Claude-4-Sonnet Gemini-2.5-Flash Gemini-2.5-Pro o4-mini MACT M3DocRAG MDocAgent SimpleDoc DocLens Qwen3-VL-235B-A22B Gemini-3-Pro-Preview

Builds on: GRPO Qwen3-VL MinerU DocLens ViDoRe-v3

Closest papers by meaning

• ColPali: Efficient Document Retrieval with Vision Language Models (ICLR 2025 · similarity 0.86)
• VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents (ICLR 2025 · similarity 0.86)
• Fix Before Search: Benchmarking Agentic Visual Query Pre-processing in Multimodal Retrieval-augmented Generation (ICML 2026 · similarity 0.86)
• Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graph (ICML 2026 · similarity 0.86)
• Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding (CVPR 2026 · similarity 0.86)
• SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding (ICLR 2025 · similarity 0.85)