Research map / Vision-language and multimodal / Human and embodied modeling
OCR-Free Document Understanding Transformer
Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, Seunghyun Park
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove OCR dependence: use end-to-end Donut instead of OCR-based LayoutLMv2 | Donut · RVL-CDIP | accuracy (%) and time (ms): 95.25 -> 95.30; 1489 -> 752 ms | Improved |
| use Donut instead of LayoutLMv2 (OCR-based) for information extraction | Donut · CORD | field-level F1 (%): 78.9 -> 84.1 | Improved |
| use Donut instead of LayoutLMv2 for information extraction | Donut · Ticket | tree-edit-distance accuracy (%): 90.1 -> 98.7 | Improved |
| use end-to-end Donut instead of OCR-based LayoutLMv2 | Donut · DocVQA test | ANLS: 78.1 -> 67.5 | Mixed |
| use Donut instead of LayoutLMv2-Large-QG on handwritten DocVQA documents | Donut · DocVQA test (handwritten) | ANLS: 67.3 -> 72.1 | Improved |
| use text reading as pre-training task instead of other tasks such as image captioning (ablation) | Donut pre-training · CORD and DocVQA | fine-tuning performance | Improved |
| add real IIT-CDIP images to synthetic SynthDoG pre-training data (ablation) | Donut pre-training · DocVQA | ANLS | Improved |
| add real IIT-CDIP images to synthetic SynthDoG pre-training data (ablation) | Donut pre-training · CORD | accuracy | No clear effect |
| use Swin Transformer or EfficientNetV2 encoder instead of other ImageNet backbones (ablation) | Donut encoder · CORD and DocVQA | performance | Improved |
| increase input resolution (ablation) | Donut · CORD and DocVQA | accuracy | Improved |
| use input resolution 1280x960 on CORD (ablation) | Donut · CORD | speed and accuracy: 0.7 sec./image and 91.1 accuracy | Mixed |
| use larger input resolution 2560x1920 in extremely low-resource training (ablation) | Donut · CORD (80 training samples) | accuracy | Improved |
Closest papers by meaning
- DocFormerv2: Local Features for Document Understanding (AAAI 2024 · similarity 0.89)
- Hierarchical Visual Feature Aggregation for OCR-Free Document Understanding (NeurIPS 2024 · similarity 0.88)
- UPOCR: Towards Unified Pixel-Level OCR Interface (ICML 2024 · similarity 0.87)
- Nougat: Neural Optical Understanding for Academic Documents (ICLR 2024 · similarity 0.86)
- InstructDoc: A Dataset for Zero-Shot Generalization of Visual Document Understanding with Instructions (AAAI 2024 · similarity 0.86)
- DocVLM: Make Your VLM an Efficient Reader (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
