Research map / Detection and segmentation / CNN features
Arbitrary Reading Order Scene Text Spotter with Local Semantics Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LSGSpotter, a detection-free auto-regressive scene text spotter that predicts start points and decodes characters with grid-sampled local attention to handle arbitrary reading order text.
Problem
Scene text spotters relying on detection-then-recognition struggle to determine reading order, especially for inverse or arbitrarily ordered text, and dense cross-attention over whole images is computationally costly.
Approach
A Start Point Localization Module predicts text start points that set reading order, and a Multi-scale Adaptive Attention Module aggregates features from adaptively sized local grids around each decoded character, so the auto-regressive decoder predicts character content and position shifts without character-level detection; reference-point disturbance during training mitigates exposure bias.</num
Key results
- Achieves 73.7% (None) and 82.3% (Full) on InverseText, 4.9% above IAST in lexicon-free setting.
- Reaches 81.5% F1 (None) on Total-Text and 68.9% on SCUT-CTW1500 without lexicon.
- Grid sampling reduces computation from 846.35G to 194.47G FLOPs (ablation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add reference point disturbance during training (ablation) | LSGSpotter · Total-Text | spotting score: 77.8 -> 81.5 (+3.7) | Helped |
| Add reference point disturbance during training (ablation) | LSGSpotter · SCUT-CTW1500 | spotting score: 62.6 -> 68.9 (+6.3) | Helped |
| Add centerline map Fc to region map Fr in SPLM (ablation) | LSGSpotter · Total-Text | F-measure: 76.4 -> 81.1 (+4.7) | Helped |
| Add start point map Fs to region map Fr in SPLM (ablation) | LSGSpotter · Total-Text | F-measure: 76.4 -> 80.4 (+4.0) | Helped |
| Use all three maps Fr, Fc, and Fs in SPLM (ablation) | LSGSpotter · Total-Text | F-measure: 76.4 -> 81.5 (+5.1) | Helped |
| Use local grid sampling instead of dense attention (ablation) | LSGSpotter MAAM · SCUT-CTW1500 | spotting score: 58.6 -> 63.7 (+5.1) | Helped |
| Add adaptive scale to local grid sampling (ablation) | LSGSpotter MAAM · SCUT-CTW1500 | spotting score: 63.7 -> 68.9 (+10.3 overall) | Helped |
| Vary adaptive grid size from 5×5 to 7×7 (ablation) | LSGSpotter MAAM · SCUT-CTW1500 | spotting score: 68.9 -> 69.2 (+0.3) | Helped |
| Remove grid sampling (ablation) | LSGSpotter · LSGSpotter | FLOPs: 194.47G -> 846.35G | Hurt |
| Set loss weight λ to 1 instead of λ=0.1 (ablation) | LSGSpotter · InverseText None | spotting score: 72.9 -> 73.7 | Helped |
| Replace predicted start points with ground-truth start points (ablation) | LSGSpotter · Total-Text None | spotting score: 81.5 -> 84.0 | Helped |
| Use LSGSpotter fine-grained localization instead of DEER single-point localization (ablation) | scene text spotter · Total-Text None | spotting score: 74.8 -> 81.5 (+6.7) | Helped |
Surprising findings
- Training with exact reference points without disturbance causes characters to be omitted at inference (77.8 vs 81.5 on Total-Text).
- Fails to detect mirror-inverted text, attributed to its limited occurrence in training data.
Limitations
- Struggles with contextless words because decoding relies on semantic context.
- Fails to spot mirror-inverted text.
- Start-point localization can miss words, limiting recall.
Experimental setup
Datasets: SynthText-150k MLT-2017 ICDAR2013 ICDAR2015 TextOCR Total-Text InverseText SCUT-CTW1500
Benchmarks: InverseText Total-Text SCUT-CTW1500 ICDAR2023-ReST
Metrics: F1-score (point-based metric, None and Full lexicon) FLOPs FPS
Baselines: ABCNet v2 SPTS SPTS v2 ESTextSpotter IAST UNITS DeepSOLO TESTR SwinTextSpotter MANGO
Builds on: SPTS PGNet DEER ESTextSpotter
Closest papers by meaning
- TextScanner: Reading Characters in Order for Robust Scene Text Recognition (AAAI 2020 · similarity 0.88)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.86)
- CLIP is Almost All You Need: Towards Parameter-Efficient Scene Text Retrieval without OCR (CVPR 2025 · similarity 0.86)
- SemiETS: Integrating Spatial and Content Consistencies for Semi-Supervised End-to-end Text Spotting (CVPR 2025 · similarity 0.86)
- LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network (AAAI 2024 · similarity 0.85)
- LayoutFormer: Hierarchical Text Detection Towards Scene Text Understanding (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.