Research map / Vision-language and multimodal / Human and embodied modeling
DocFormer: End-to-End Transformer for Document Understanding
Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, R. Manmatha
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| share spatial embeddings across vision and text (ablation) | DocFormer · FUNSD | F1: 76.9 vs 75.58 (-1.32 without sharing) | Improved |
| share spatial embeddings across vision and text (ablation) | DocFormer · CORD | F1: 93.36 vs 92.51 (-0.85 without sharing) | Improved |
| add unsupervised pre-training before fine-tuning (ablation) | DocFormer-base · FUNSD | F1: 4.18 -> 83.34 | Improved |
| add unsupervised pre-training before fine-tuning (ablation) | DocFormer-base · CORD | F1: 0.54 -> 96.33 | Improved |
| add unsupervised pre-training before fine-tuning (ablation) | DocFormer-base · RVL-CDIP | accuracy: 93.95 -> 96.17 | Improved |
| use a deeper projection head instead of linear (ablation) | DocFormer-base · FUNSD (149 train samples) | F1: 83.34 -> 82.93 (-0.41) | Hurt |
| use a deeper projection head instead of linear (ablation) | DocFormer-base · CORD (800 train samples) | F1: 96.33 -> 96.87 (+0.54) | Improved |
| use a deeper projection head instead of linear (ablation) | DocFormer-base · RVL-CDIP | accuracy: 96.17 -> 96.85 (+0.68) | Improved |
| add multi-modal MLM to MLM pre-training (ablation) | DocFormer · FUNSD (1M-doc pretraining) | F1: 72.40 -> 73.91 (+1.51) | Improved |
| add Learn to Reconstruct pre-training task (ablation) | DocFormer · FUNSD (1M-doc pretraining) | F1: 73.91 -> 74.68 (+0.77) | Improved |
| add Learn to Reconstruct pre-training task (ablation) | DocFormer · CORD (1M-doc pretraining) | F1: 90.98 -> 92.61 (+1.63) | Improved |
| add Text describes Image pre-training task (ablation) | DocFormer · FUNSD (1M-doc pretraining) | F1: 74.68 -> 76.90 (+2.23) | Improved |
Closest papers by meaning
- DocFormerv2: Local Features for Document Understanding (AAAI 2024 · similarity 0.94)
- HRVDA: High-Resolution Visual Document Assistant (CVPR 2024 · similarity 0.88)
- DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding (CVPR 2025 · similarity 0.87)
- InstructDoc: A Dataset for Zero-Shot Generalization of Visual Document Understanding with Instructions (AAAI 2024 · similarity 0.87)
- Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Models (CVPR 2024 · similarity 0.86)
- Docopilot: Improving Multimodal Models for Document-Level Understanding (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
