Research map / Vision-language and multimodal / Multimodal LLMs
CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy
Zhibo Yang, Jun Tang, Zhaohai Li, Pengfei Wang, Jianqiang Wan, Humen Zhong, Xuejing Liu, Mingkun Yang, Peng Wang, Shuai Bai, Lianwen Jin, Junyang Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| generalist LMMs versus specialist OCR models | Multi-Scene OCR track · CC-OCR Multi-Scene OCR | total score: 83.25 (Gemini-1.5-Pro) vs 61.00 (best specialist GOT) | Improved |
| evaluate natural scenes instead of documents | LMMs · CC-OCR Multi-Scene OCR | score: about 15% lower | Hurt |
| rotate images by 0, 90, 180 or 270 degrees | LMMs · CC-OCR Multi-Scene OCR | total score: Gemini-1.5-Pro 83.25 -> 79.45; InternVL2-76B 76.92 -> 38.00 | Hurt |
| ask for text grounding output instead of recognition only | LMMs · CC-OCR English multi-scene subsets | score: Gemini-1.5-Pro 60.98; others below 50 | Hurt |
| evaluate multilingual instead of English and Chinese text | LMMs · CC-OCR Multilingual OCR | score | Hurt |
| evaluate Arabic, Japanese, Korean, Vietnamese instead of Latin-script languages | LMMs · CC-OCR Multilingual OCR | score | Hurt |
| require structured output (parsing, KIE) instead of plain text recognition | LMMs · CC-OCR Document Parsing and KIE | score: best parsing 62.37, best KIE 71.76 | Hurt |
| evaluate handwritten formulas | LMMs · CC-OCR Document Parsing | score | Hurt |
| evaluate Chinese instead of English documents | LMMs · CC-OCR Document Parsing | score: Gemini-1.5-Pro about 5% lower on Chinese tables | Hurt |
Closest papers by meaning
- OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning (NeurIPS 2025 · similarity 0.89)
- PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks (CVPR 2026 · similarity 0.88)
- MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios (NeurIPS 2025 · similarity 0.86)
- OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoning (ICLR 2026 · similarity 0.86)
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.86)
- Seeing is Believing? Mitigating OCR Hallucinations in Multimodal Large Language Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
