Research map / Vision-language and multimodal / Human and embodied modeling
MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MPDocBench-Parse, a 433-document, 3,246-page bilingual benchmark for end-to-end multi-page document parsing with eight metrics covering truncated text/table merging, figure extraction, reading order, and heading hierarchy, and benchmarks 19 specialized and general VLMs.
Problem
Existing document parsing benchmarks are mostly single-page, text-centric, and academic-biased, and do not evaluate cross-page semantic continuity, heading hierarchy, or figure extraction in realistic multi-page documents.
Approach
A manually annotated multi-page benchmark with structural relations (parent-child, truncated text/table) and eight-dimensional evaluation protocol, including truncated text/table merging, figure extraction F1, and tree-edit-distance heading hierarchy scoring, run end-to-end on full documents.
Key results
- PaddleOCR-VL-1.5 (pipeline) achieves the best overall score of 80.80, versus 73.98 for the best general VLM Qwen3-VL-235B.
- Cross-page truncated text merging is failed by nearly all models, with PaddleOCR-VL-1.5 and MinerU2.5 the only strong performers on cross-page table merging (83.07 and 88.14 overall).
- Heading hierarchy recovery is the weakest dimension; the best score is 50.84 (FireRed-OCR), and general VLMs score as low as 26.94 (Gemini-3.1-pro-preview).
Surprising findings
- Specialized pipeline and end-to-end parsers outperform general VLMs on every one of the eight tasks, with no general VLM ranking first on any task.
- Figure extraction for InternVL-3.5-38B collapses to 7.97 F1, and Gemini-3.1-pro-preview ranks low overall despite strong general capability.
Limitations
- TEDS for table merging can give high scores even when merged structure is incomplete or partly wrong.
- Heading hierarchy scoring relies on Markdown '#' prefixes, which may not fully reflect true hierarchy quality for some models.
- Documents were collected from the internet and the benchmark is limited to 15 document types in English and Chinese.
Experimental setup
Datasets: MPDocBench-Parse
Benchmarks: MPDocBench-Parse
Metrics: NED (text, reading order) CDM (formula) TEDS (table, heading hierarchy) truncated text NED truncated table TEDS figure F1 overall average
Baselines: GLM-OCR PaddleOCR-VL-1.5 MinerU2.5 Youtu-Parsing MonkeyOCR-pro-3B Dolphin-v2 dots.mocr FireRed-OCR dots.ocr DeepSeek-OCR2 OCRVerse Logics-Parsing-v2
Builds on: OmniDocBench READoc PP-DocLayoutV2
Closest papers by meaning
- MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios (NeurIPS 2026 · similarity 0.90)
- Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training (CVPR 2026 · similarity 0.89)
- OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations (CVPR 2025 · similarity 0.89)
- Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation (NeurIPS 2026 · similarity 0.88)
- DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translation (NeurIPS 2026 · similarity 0.87)
- MMLONGBENCH-DOC: Benchmarking Long-context Document Understanding with Visualizations (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.