Research map / Label-efficient and robust learning / Semi-supervised learning
SemiETS: Integrating Spatial and Content Consistencies for Semi-Supervised End-to-end Text Spotting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SemiETS, a teacher-student semi-supervised framework for end-to-end text spotting that uses progressive hierarchical pseudo-label assignment and mutual mining between detection and recognition.
Problem
Semi-supervised end-to-end text spotting is under-explored; directly applying general-scene SSL methods suffers from inconsistent pseudo labels between detection and recognition and from ambiguous supervision caused by teacher/student misalignment.
Approach
Progressive Sample Assignment builds hierarchical Det-only and E2E pseudo labels via a joint constraint filter and recognition filter; Mutual Mining Strategy uses polygon DIoU to reweight recognition loss (spatial-aware) and Levenshtein distance between teacher/student transcriptions to amplify detection loss (content-aware).
Key results
- Outperforms prior SSL SOTA by 8.7% E2E (None) on Total-Text with 0.5% labeled data.
- Improves the fully supervised DeepSolo baseline on Total-Text E2E (None) by 2.0% (79.7 to 81.7).
- Domain adaptation from Total-Text to IC15 improves detection and E2E (generic lexicon) by 17.3% and 14.4%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SemiETS instead of STAC* | DeepSolo · Total-Text, 0.5% labeled, None | E2E H-mean: 63.3 -> 72.0 | Helped |
| Use SemiETS instead of Mean-Teacher* | DeepSolo · CTW1500, 0.5% labeled, None | E2E H-mean: 48.8 -> 53.5 | Helped |
| Use SemiETS instead of Semi-DETR* | DeepSolo · ICDAR 2015, 0.5% labeled, G | E2E H-mean: 65.3 -> 67.8 | Helped |
| Add Progressive Sample Assignment (ablation) | SemiETS on Total-Text, 2% labeled · Total-Text, 2% labeled | E2E H-mean (None): 68.6 -> 72.4 | Helped |
| Add Mutual Mining Strategy without PSA (ablation) | DeepSolo on Total-Text, 2% labeled · Total-Text, 2% labeled | E2E H-mean (None): 68.6 -> 71.3 | Helped |
| Add PSA and MMS together (ablation) | DeepSolo on Total-Text, 2% labeled · Total-Text, 2% labeled | E2E H-mean (None): 68.6 -> 73.4 | Helped |
| Add recognition threshold and confidence comparison in PSA (ablation) | DeepSolo on Total-Text, 2% labeled · Total-Text, 2% labeled | Detection F1: 72.5 -> 77.9 | Helped |
| Add hybrid matching to PSA (ablation) | DeepSolo on Total-Text, 2% labeled · Total-Text, 2% labeled | Detection F1: 77.9 -> 81.3 | Helped |
| Add Spatial-aware Consistency Integration (ablation) | DeepSolo with PSA on Total-Text, 2% labeled · Total-Text, 2% labeled | E2E H-mean (None): 72.4 -> 73.3 | Helped |
| Add Content-aware Region Calibration (ablation) | DeepSolo with PSA on Total-Text, 2% labeled · Total-Text, 2% labeled | E2E H-mean (None): 72.4 -> 73.3 | Helped |
| Add SCI and CRC together (ablation) | DeepSolo with PSA on Total-Text, 2% labeled · Total-Text, 2% labeled | E2E H-mean (None): 72.4 -> 73.4 | Helped |
| Replace IoU with polygon DIoU in SCI (ablation) | DeepSolo on Total-Text, 2% labeled · Total-Text, 2% labeled | E2E H-mean (None): 72.8 -> 73.3 | Helped |
Surprising findings
- Most EMA-based SSL methods (Mean-Teacher, Soft Teacher, Unbiased Teacher v2) degrade under low label proportions, which the authors attribute to limited model capacity introducing cumulative error.
- Generalist VLLMs such as InternVL2-8B and Qwen2-VL-7B perform near zero on text spotting benchmarks.
Limitations
- Focuses on Latin text; multilingual spotting not explored.
- Usage of text characteristics and detection-recognition synergy is not explored deeply.
Experimental setup
Datasets: Total-Text ICDAR 2015 CTW1500 TextOCR Synth150K
Benchmarks: Total-Text ICDAR 2015 CTW1500
Metrics: H-mean (E2E F1 with None/Weak/Generic/Strong lexicon) Detection F1 Precision Recall Word spotting F1
Baselines: STAC Mean-Teacher Soft Teacher Unbiased Teacher v2 Semi-DETR Supervised DeepSolo ABCNet
Builds on: DeepSolo Semi-DETR Mean-Teacher STAC Hybrid matching
Closest papers by meaning
- GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matching (NeurIPS 2024 · similarity 0.88)
- Bridging the Gap Between End-to-End and Two-Step Text Spotting (CVPR 2024 · similarity 0.87)
- Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing (NeurIPS 2024 · similarity 0.86)
- Arbitrary Reading Order Scene Text Spotter with Local Semantics Guidance (AAAI 2025 · similarity 0.86)
- Learning Textual Prompts for Open-World Semi-Supervised Learning (CVPR 2025 · similarity 0.85)
- LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.