Research map / Vision-language and multimodal / Human and embodied modeling
OTE: Exploring Accurate Scene Text Recognition Using One Token
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OTE, a scene text recognizer that encodes each text image into a single global token with a ViT-based image-to-vector encoder and decodes characters from it via a vector-to-sequence decoder (AR or NAR), also applied to scene text retrieval.
Problem
Scene text recognition typically relies on sequence-to-sequence decoders over visual token sequences, which suffer from attention drift or CTC blank/repeat alignment issues; the paper asks whether a single vector can represent a text image for accurate recognition.
Approach
Replace the visual token sequence with one global token from a ViT/SVTR encoder (CLS token or global average pooling over multi-grained features) and decode characters by diffusing this vector to character positions via channel-wise parallel attention, followed by a masked self-attention lingual decoder that supports both autoregressive and non-autoregressive decoding.
Key results
- OTE with ViT-B (autoregressive) reaches 93.5% average word accuracy on six regular/irregular STR benchmarks trained on MJ+ST, the best among compared methods including language-aware ones.
- OTE achieves 80.90 mAP on IIIT-STR retrieval, a 3.81-point improvement over the prior best (Wang et al., 77.09).
- On Union14M-L training, OTE with SVTR (autoregressive) gets 77.9% average across Union14M-Benchmark subsets, with strong results on curve, multi-oriented and salient subsets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ViT-B instead of ViT-S as the backbone (ablation) | OTE · IIIT, SVT, IC13, IC15, SVTP, CUTE (MJ+ST) | word accuracy: 92.8 -> 93.5 | Helped |
| Use SVTR instead of ViT-S as the backbone (ablation) | OTE · IIIT, SVT, IC13, IC15, SVTP, CUTE (MJ+ST) | word accuracy: 92.8 -> 93.4 | Helped |
| Replace CLS token aggregation with global average pooling (ablation) | OTE / ViT-S with autoregressive decoding · IIIT, SVT, IC13, IC15, SVTP, CUTE (MJ+ST) | word accuracy: 92.8 -> 92.5 | No clear effect |
| Use autoregressive decoding instead of non-autoregressive decoding (ablation) | OTE / ViT-S · IIIT, SVT, IC13, IC15, SVTP, CUTE (MJ+ST) | word accuracy: 92.2 -> 92.8 | Helped |
| Use OTE with SVTR instead of ParseqA | Scene text recognition · Uber | word accuracy: 42.0 -> 47.8 | Helped |
| Use OTE with SVTR instead of ParseqA | Scene text recognition · ArT | word accuracy: 70.7 -> 69.1 | Hurt |
| Use OTE with SVTR instead of ParseqA | Scene text recognition · COCO | word accuracy: 64.0 -> 64.5 | Helped |
| Use OTE instead of Wang et al. | Scene text retrieval · IIITSTR | mAP: 77.09 -> 80.90 | Helped |
| Use OTE / SVTR instead of the strongest listed baseline | Scene text recognition trained on Union14M-L · Union14M-Benchmark (Avg) | word accuracy: 78.6 -> 77.9 | Hurt |
Surprising findings
- A single global token suffices for accurate recognition, outperforming many sequence-based methods without language modeling or post-processing.
- Global average pooling over visual tokens performs comparably to a dedicated CLS token for the global representation.
Limitations
- Performance degrades on words of extremely long length, since compressing long text into one token is difficult.
- Weaker on multi-word text than SOTA, attributed to limited fine-grained backbone capacity.
Experimental setup
Datasets: MJ ST Union14M-L MLT-5k IIIT5K SVT IC13 IC15 SVTP CUTE ArT COCO-Text
Benchmarks: IIIT5K SVT IC13 IC15 SVTP CUTE ArT COCO-Text Uber-Text Union14M-Benchmark IIIT-STR
Metrics: word accuracy mAP
Baselines: CRNN SVTR ABINet PARSeq MAERec MATRN CONCLR MGP Wang et al. (retrieval) Gomez et al.
Builds on: Vision Transformer SVTR CLIP SENet PARSeq
Closest papers by meaning
- Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition (CVPR 2025 · similarity 0.89)
- Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing (NeurIPS 2024 · similarity 0.88)
- CLIP is Almost All You Need: Towards Parameter-Efficient Scene Text Retrieval without OCR (CVPR 2025 · similarity 0.88)
- Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation (CVPR 2025 · similarity 0.88)
- ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining (AAAI 2024 · similarity 0.87)
- Image as a Language: Revisiting Scene Text Recognition via Balanced, Unified and Synchronized Vision-Language Reasoning Network (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.