Research map / Label-efficient and robust learning / Recognition and long-tail
Sequence-to-Sequence Contrastive Learning for Text Recognition
Aviad Aberdam, Ron Litman, Shahar Tsiper, Oron Anschel, Ron Slossberg, Shai Mazor, R. Manmatha, Pietro Perona
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace whole-image SimCLR with sequence-level SeqCLR | text recognition representation learning · IAM, RIMES, CVL, IIT5K, IC03, and IC13 decoder evaluation | accuracy | Improved |
| use window-to-instance rather than frame-to-instance mapping (ablation) | SeqCLR · scene-text decoder evaluation | accuracy: 35.7 vs 3.0; 43.6 vs 4.6; 43.5 vs 4.9 | Improved |
| use window-to-instance rather than all-to-instance mapping (ablation) | SeqCLR · scene-text decoder evaluation with CTC | accuracy: 35.7 vs 20.4; 43.6 vs 24.2; 43.5 vs 24.4 | Improved |
| use SeqCLR pretraining rather than supervised-only training | text recognizer · handwritten datasets with limited labeled data | accuracy: IAM 5% CTC: 21.4 -> 31.2; RIMES 5% CTC: 35.9 -> 61.8 | Improved |
| use SeqCLR pretraining and fine-tune with all labels rather than supervised-only training | text recognizer · IAM and RIMES handwritten recognition | accuracy: IAM attention: 77.8 -> 79.9; RIMES attention: 89.3 -> 92.5 | Improved |
| use SeqCLR pretraining rather than supervised-only training | text recognizer with attention decoder · scene-text datasets | accuracy | No clear effect |
| use SeqCLR rather than the strongest listed word-level method | handwritten text recognition · IAM | WER: 22.2 -> 20.1 | Improved |
| use SeqCLR rather than the strongest listed word-level method | handwritten text recognition · RIMES | WER: 9.6 -> 7.6 | Improved |
| add a BiLSTM projection head (ablation) | SeqCLR All-to-instance · RIMES representation evaluation, contextual features, CTC | accuracy: 55.4 -> 59.0 | Improved |
| add an MLP-per-frame projection head (ablation) | SeqCLR Frame-to-instance · RIMES representation evaluation, contextual features, CTC | accuracy: 49.9 -> 57.5 | Improved |
| add a BiLSTM projection head (ablation) | SeqCLR Window-to-instance · RIMES representation evaluation, contextual features, attention | accuracy: 79.5 -> 75.9 | Hurt |
| add a BiLSTM projection head (ablation) | SeqCLR All-to-instance · RIMES representation evaluation, visual features, CTC | accuracy: 2.0 -> 35.5 | Improved |
Closest papers by meaning
- Sequence Level Contrastive Learning for Text Summarization (AAAI 2022 · similarity 0.87)
- Towards the Unseen: Iterative Text Recognition by Distilling From Errors (ICCV 2021 · similarity 0.87)
- Retrieval-Enhanced Contrastive Vision-Text Models (ICLR 2024 · similarity 0.86)
- Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Models (CVPR 2024 · similarity 0.86)
- Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing (NeurIPS 2024 · similarity 0.86)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
