Research map / Label-efficient and robust learning / Semi-supervised learning
Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ViSu, a semi-supervised scene text recognition framework that combines Mean Teacher with real unlabeled data, an Online Generation Strategy for background-free stylized samples, and a Character Unidirectional Alignment loss that corrects a formula error in the character contrastive loss.
Problem
Scene text recognition models struggle with artistic and severely distorted characters because synthetic training data is simple and monotonous and models are sensitive to character morphology.
Approach
Mean-teacher semi-supervised STR with (i) online generation of background-free, randomly styled text samples from synthetic labels used as teacher inputs, and (ii) a unidirectional alignment loss that aligns strong-augmentation student character features to teacher base-image features, with the derivation error of the prior character contrastive loss corrected.
Key results
- ViSu reaches 94.7% weighted average accuracy on six common benchmarks and 70.9% average on Union14M-Benchmark when trained on synthetic plus real unlabeled data.
- Compared with the reproduced TRBA-cr under the same data, ViSu gains 11.7% on Union-B and 27.9% on Uber-Text per the main comparison.
- Trained with real labeled Union14M-L plus unlabeled data, ViSu reaches 86.9% average on Union14M-Benchmark, above MAERec-B's 85.2% with about 1/5 the parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ViSu training with real unlabeled data, OGS, URF, and CUA Loss | Baseline · Union14M-Benchmark | AVG accuracy: 62.6 -> 70.9 | Helped |
| Add OGS samples to 10% synthetic training data (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 58.2 -> 60.3 | Helped |
| Train using OGS samples only instead of combining them with synthetic data (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 60.3 -> 8.2 | Hurt |
| Add CUA Loss rather than using no alignment loss (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 68.2 -> 70.9 | Helped |
| Replace CUA Loss with CC¹ Loss (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 70.9 -> 69.1 | Hurt |
| Align strong augmented images to base images using CC² rather than CUA Loss (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 70.9 -> 70.1 | Hurt |
| Remove URF (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 70.9 -> 61.4 | Hurt |
| Remove real unlabeled data (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 70.9 -> 64.5 | Hurt |
| Remove OGS (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 70.9 -> 68.8 | Hurt |
| Use CE instead of KL-divergence for consistency loss (ablation) | ViSu · Union14M-Benchmark | AVG accuracy: 70.8 vs 70.9 | No clear effect |
| Add random background to OGS samples (ablation) | ViSu OGS · Union14M-Benchmark | AVG accuracy: 70.9 -> 69.5 | Hurt |
| Add random text color to OGS samples (ablation) | ViSu OGS · Union14M-Benchmark | AVG accuracy: 70.9 -> 70.7 | Hurt |
Surprising findings
- A model trained only on OGS samples (without simple synthetic data) reaches just 8.2% average accuracy on Union-B, showing that challenging generated data alone cannot bootstrap recognition.
Limitations
- Struggles with images of extreme aspect ratio because fixed-size resizing distorts long text.
- Relies on synthetic labeled data and real unlabeled data; the real unlabeled set (Union14M-U) is not fully publicly available for the TRBA-cr comparison.
Experimental setup
Datasets: MJSynth SynthText Union14M-U Book32 CC OpenImages IIIT SVT IC13 IC15 SVTP CUTE
Benchmarks: IIIT SVT IC13 IC15 SVTP CUTE Union14M-Benchmark WordArt ArT COCO-Text Uber-Text
Metrics: word accuracy weighted average accuracy
Baselines: TRBA-cr ParSeq CLIP-OCR LPV MGP LISTER ABINet MAERec-B
Builds on: Mean Teacher TRBA Character Contrastive Loss ParSeq
Closest papers by meaning
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.88)
- DreamText: High Fidelity Scene Text Synthesis (CVPR 2025 · similarity 0.88)
- Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition (CVPR 2025 · similarity 0.87)
- SemiETS: Integrating Spatial and Content Consistencies for Semi-Supervised End-to-end Text Spotting (CVPR 2025 · similarity 0.86)
- TextNeRF: A Novel Scene-Text Image Synthesis Method based on Neural Radiance Fields (CVPR 2024 · similarity 0.86)
- Layout-Agnostic Scene Text Image Synthesis with Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.