Research map / Vision-language and multimodal / Multimodal LLMs

Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DocHumming, a 1B end-to-end document parser trained on synthesized full-page data (DocMix-3M via Realistic Scene Synthesis) with progressive two-stage training and structure-token loss weighting, plus a real-world captured benchmark Wild-OmniDocBench.

Problem

End-to-end MLLM document parsing produces repetitive, hallucinated, and structurally inconsistent outputs, especially on casually captured documents, due to scarce full-page end-to-end training data and lack of structure-aware training.

Approach

Synthesize large-scale full-page parsing data by composing atomic elements into layout templates with reading order and capture-style augmentation, train progressively from element-level to page-level inputs, and upweight loss on structure tokens (e.g., table tags).

Key results

• DocHumming (1B) reaches 93.75 Overall on OmniDocBench, above PaddleOCR-VL (91.93) and Qwen3-VL-235B (89.15).
• On Wild-OmniDocBench, DocHumming scores 87.03 Overall, versus 79.69 for Qwen3-VL-235B and 78.01 for dots.ocr.
• Ablations show RSS adds +3.79 Overall on OmniDocBench and +8.21 on Wild-OmniDocBench versus a PDF-to-LaTeX baseline, and repetition rate drops from 4.7 to 2.1 on OmniDocBench.

Surprising findings

• Synthetic DocMix-1M underperforms the 100K manually annotated data on both benchmarks, and only surpasses it at about 3M pages.
• Performance saturates near 3M synthetic pages, with DocMix-4M giving marginal changes, attributed to limited element and template diversity.

Limitations

• Performance drops on irregular, interleaved layouts such as newspapers and posters.
• Ultra-high-resolution pages require downsampling or tiling, which can induce repeated or missing content.
• Parsing text-dense pages takes about 3 s per page, limiting interactive use.

Experimental setup

Datasets: OmniDocBench XFUND DocMix-3M 100K manually annotated real documents table recognition datasets formula parsing datasets paragraph datasets public layout datasets with reading order

Benchmarks: OmniDocBench XFUND Wild-OmniDocBench

Metrics: Overall Text Edit distance Formula CDM Table TEDS Reading Order Edit Repetition rate

Baselines: PP-StructureV3 MinerU2.5 PaddleOCR-VL Qwen3-VL-235B Qwen2.5-VL-72B Gemini-2.5 Pro GPT-4o DeepSeek-OCR dots.ocr MonkeyOCR-pro-3B Mistral OCR POINTS-Reader

Builds on: InternVL2 OmniDocBench GOT SynthDog DocLayout-YOLO

Closest papers by meaning

• DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translation (NeurIPS 2026 · similarity 0.90)
• MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing (NeurIPS 2026 · similarity 0.89)
• MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios (NeurIPS 2026 · similarity 0.88)
• Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation (NeurIPS 2026 · similarity 0.88)
• Boosting Document Parsing Efficiency and Performance with Coarse-to-Fine Visual Processing (CVPR 2026 · similarity 0.87)
• DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding (CVPR 2025 · similarity 0.86)