AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

OvisOCR: End-to-End Document Parsing via Aligning Specialized Perception with General Reasoning

Jun-Peng Jiang, Shiyin Lu, An-Yang Ji, Yinglun Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes OvisOCR, a lightweight end-to-end MLLM that maps full-page document images directly to structured Markdown without crop-OCR-merge cascades or layout detection.

Problem

Document parsing of high-resolution pages typically relies on complex Crop-OCR-Merge cascades with layout detection; the paper targets strictly end-to-end parsing of full-page images to Markdown.

Approach

Directly maps full-page visual signals to structured Markdown with a lightweight MLLM, avoiding localized slicing and layout detection dependencies, and aligns specialized perception with general reasoning.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add GRPO with a single global markdown edit-distance reward (ablation)Ovis-SFT · OmniDocBench v1.5Overall: 92.30 -> 92.35No clear effect
replace global reward with decoupled per-category rewards (ablation)Ovis-GRPO-MD · OmniDocBench v1.5Overall: 92.35 -> 92.84Improved
add decoupled rewards (vs SFT) (ablation)Ovis-SFT · OmniDocBench v1.5FormulaCDM: 90.40 -> 90.70Improved
add decoupled rewards (vs global reward) (ablation)Ovis-GRPO-MD · OmniDocBench v1.5TableTEDS: 89.69 -> 91.03Improved
add Semantic Auditor filtered data (ablation)Ovis-GRPO-DR · OmniDocBench v1.5Overall: 92.84 -> 93.07Improved
add Semantic Auditor filtered data (ablation)Ovis-GRPO-DR · OmniDocBench v1.5FormulaCDM: 90.70 -> 91.61Improved
add Semantic Auditor filtered data (ablation)Ovis-GRPO-DR · OmniDocBench v1.5TableTEDS: 91.03 -> 90.80Hurt
add category-wise advantage normalization (GDPO) (ablation)Ovis-GRPO-SA · OmniDocBench v1.5Overall: 93.07 -> 93.19Improved
add full RL alignment (GDPO) to SFT model (ablation)Ovis-SFT · OmniDocBench v1.5Overall: 92.30 -> 93.19Improved

Experimental setup

Benchmarks: OmniDocBench

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy