Research map / Vision-language and multimodal / Human and embodied modeling
Instruction-Guided Scene Text Recognition
Yongkun Du, Zhineng Chen, Yuchen Su, Caiyan Jia, Yu–Gang Jiang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add frequency, position and sub-string attribute questions to recognition instructions (ablation) | IGTR-AR · Union14M-L test (Union14M-L-trained) | accuracy: 81.27 -> 84.02 | Improved |
| add all attribute questions (full instructions) (ablation) | IGTR-PR · Union14M-L test (Union14M-L-trained) | accuracy: +1.94% | Improved |
| add condition modes cond-1 to cond-4 (ablation) | IGTR-PR · Union14M-L test (Union14M-L-trained) | accuracy: 79.02 -> 82.23 | Improved |
| increase number of partitions K from 2 to 6 (ablation) | IGTR-AR · Union14M-L test (Union14M-L-trained) | accuracy: 82.41 -> 83.87 | Improved |
| add query-to-condition and query-to-image cross-attention stages (ablation) | IGTR-AR · Union14M-L test (Union14M-L-trained) | accuracy: 81.23 -> 84.86 | Improved |
| add question-to-condition and question-to-image stages (ablation) | IGTR-AR · Union14M-L test (Union14M-L-trained) | accuracy: 82.01 -> 84.86 | Improved |
| train 60 epochs instead of 20 with instruction sampling (ablation) | IGTR-PR · Union14M-L test (Union14M-L-trained) | accuracy: 82.51 -> 85.29 | Improved |
| train 60 epochs instead of 20 with PR instruction only (ablation) | Vanilla-PR · Union14M-L test (Union14M-L-trained) | accuracy: 76.14 -> 77.05 | Improved |
| scale image encoder from 24M to 40M (SVTR-L) (ablation) | IGTR-PR · Union14M-L test (Union14M-L-trained) | accuracy: 82.51 -> 83.60 | Improved |
| scale image encoder from 24M to 40M (SVTR-L) (ablation) | Vanilla-PR · Union14M-L test (Union14M-L-trained) | accuracy: 76.14 -> 76.23 | No clear effect |
| add targeted strengthening sampling for rare characters | IGTR-PR · CTR rare characters (1-10 occurrences) | accuracy: 63.88 -> 67.47 | Improved |
| add targeted strengthening (TS) | IGTR-AR · CTR Chinese benchmark | avg accuracy: 76.37 -> 77.17 | Improved |
Closest papers by meaning
- Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition (AAAI 2022 · similarity 0.90)
- Revisiting Scene Text Recognition: A Data Perspective (ICCV 2023 · similarity 0.89)
- InstructOCR: Instruction Boosting Scene Text Spotting (AAAI 2025 · similarity 0.89)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.89)
- Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition (CVPR 2025 · similarity 0.88)
- Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
