Research map / Vision-language and multimodal / Human and embodied modeling
TrOCR: Transformer-Based Optical Character Recognition with Pre-trained Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BEiT BASE instead of DeiT BASE as the image encoder (ablation) | TrOCR with RoBERTa BASE decoder · SROIE | F1: 69.17 -> 76.31 | Helped |
| use RoBERTa LARGE instead of RoBERTa BASE as the decoder (ablation) | TrOCR with BEiT BASE encoder · SROIE | F1: 76.31 -> 79.36 | Helped |
| replace BEiT BASE with ResNet50 as the image encoder (ablation) | TrOCR with RoBERTa BASE decoder · SROIE | F1: 76.31 -> 67.02 | Hurt |
| add pretrained model initialization (ablation) | TrOCR trained on SROIE · SROIE | F1: 38.24 -> 72.75 | Helped |
| add data augmentation (ablation) | TrOCR with pretrained initialization · SROIE | F1: 72.75 -> 82.30 | Helped |
| add first-stage pretraining (ablation) | TrOCR with data augmentation · SROIE | F1: 82.30 -> 95.48 | Helped |
| add second-stage pretraining (ablation) | TrOCR with first-stage pretraining · SROIE | F1: 95.48 -> 95.84 | Helped |
| fine-tune on synthetic data plus benchmark training sets instead of synthetic data only | TrOCR LARGE · Scene text benchmarks | word accuracy | Helped |
Closest papers by meaning
- UPOCR: Towards Unified Pixel-Level OCR Interface (ICML 2024 · similarity 0.87)
- General Detection-based Text Line Recognition (NeurIPS 2024 · similarity 0.84)
- MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios (NeurIPS 2025 · similarity 0.83)
- Enhanced Optical Character Recognition by Optical Sensor Combined with BERT and Cosine Similarity Scoring (Student Abstract) (AAAI 2024 · similarity 0.83)
- SRFormer: Text Detection Transformer with Incorporated Segmentation and Regression (AAAI 2024 · similarity 0.83)
- UNIT: Unifying Image and Text Recognition in One Vision Encoder (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.