Research map / Large language models / Retrieval and knowledge
OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation
Junyuan Zhang, Qintong Zhang, Bin Wang, Linke Ouyang, Zichen Wen, Ying Li, Ka-Ho Chow, Conghui He, Wentao Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a VLM (Qwen2.5-VL-72B) for OCR instead of pipeline MinerU | OCR for RAG knowledge base · OHRBench overall | F1: 30.0 -> 31.1 | Improved |
| Use best OCR instead of ground truth structured data | RAG system · OHRBench overall | F1: 36.1 -> 31.1 | Hurt |
| Use VLM OCR instead of pipeline OCR for reading-order questions | Qwen2.5-VL-72B · OHRBench reading order (overall) | F1: 17.8 -> 8.8 | Hurt |
| Increase semantic noise from mild to severe (ablation) | RAG retrievers and LLMs · OHRBench perturbed data | performance: nearly 50% decline | Hurt |
| Add severe formatting noise (ablation) | BGE-M3 retrieval and Llama3.1-8B generation · OHRBench multimodal questions | performance: 12.7% and 9.1% | Hurt |
| Use larger LLM (Qwen2-72B) under formatting noise (ablation) | generator · OHRBench multimodal questions | performance: 7% reduction | Improved |
| Use HTML instead of Markdown/LaTeX table format (ablation) | retrieval · OHRBench table questions | retrieval | Hurt |
| Use Markdown tables instead of HTML/LaTeX for generation (ablation) | LLM generation · OHRBench table questions | generation | Hurt |
| Replace Llama3.1-8B generator with Ext2Gen-8B-R2 (ablation) | RAG generator · OHRBench overall (Azure OCR) | F1: 24.0 -> 42.8 | Improved |
| Use detailed instead of simple OCR prompt (ablation) | VLM OCR · OHRBench | OCR/RAG performance | Improved |
| Rewrite questions by hand (Sim2Real) (ablation) | MinerU · OHRBench 100 Q&As | F1: 20.7 -> 18.0 | Hurt |
| Use Qwen2-72B instead of Qwen2-7B as generator (ablation) | error analysis with formatting noise · OHRBench | share of OCR-error but correct answers: 9.4% and 3.5% higher | Improved |
Closest papers by meaning
- VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents (ICLR 2025 · similarity 0.89)
- RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering (NeurIPS 2025 · similarity 0.87)
- AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generation (NeurIPS 2026 · similarity 0.87)
- Benchmarking Large Language Models in Retrieval-Augmented Generation (AAAI 2024 · similarity 0.87)
- RAGGED: Towards Informed Design of Scalable and Stable RAG Systems (ICML 2025 · similarity 0.87)
- VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
