Research map / Vision-language and multimodal / CLIP and zero-shot
CLIPTER: Looking at the Bigger Picture in Scene Text Recognition
Aviad Aberdam, David Bensaid, Alona Golts, Roy Ganz, Oren Nuriel, Royee Tichauer, Shai Mazor, Ron Litman
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CLIPTER scene-level CLIP features via gated cross-attention | PARSeq · 12 scene text benchmarks | weighted average word accuracy: +0.8% | Improved |
| add CLIPTER to ABINet-Vis | ABINet-Vis · 12 scene text benchmarks | weighted average word accuracy: +1.7% | Improved |
| add CLIPTER on out-of-vocabulary words | PARSeq · OOV benchmark | word accuracy (%): 68.97 -> 71.45 | Improved |
| train with CLIPTER at 10% of data versus baseline at 25% | PARSeq · low-data regime | word accuracy | Improved |
| add CLIPTER to GLASS+PARSeq two-stage spotter | GLASS + PARSeq · ICDAR 2015 E2E | E2E score: 77.3 -> 77.4 | Improved |
| use vision-language image encoder (CLIP/BLIP/GIT) instead of vision-only (DINO/ViT-MAE/OWL-ViT) (ablation) | PARSeq CLIPTER · 12 scene text benchmarks | average word accuracy gain: +0.44-0.55 -> +0.53-0.73 | Improved |
| pool scene features to a single vector (k=inf) for BLIP in TRBA (ablation) | TRBA CLIPTER · 12 scene text benchmarks | average word accuracy gain: +0.60 (k=10) -> +0.15 (k=inf) | Hurt |
| vary integration point (vision, contextual, decoder) (ablation) | CLIPTER · 12 scene text benchmarks | average word accuracy gain: ABINet Decoder +0.49 vs Contextual +0.18; ABINet-Vis Vision +1.73 vs Contextual +1.14 | Mixed |
| use more complex multi-head cross-attention fusion instead of gated attention (ablation) | CLIPTER on PARSeq · 12 scene text benchmarks | average word accuracy gain: +0.53 (gated) vs +0.55 (MH-CA mini) | No clear effect |
| scale fusion module from MH-CA mini to small on TRBA (ablation) | CLIPTER on TRBA · 12 scene text benchmarks | average word accuracy gain: +0.67 -> +0.54 | Hurt |
Closest papers by meaning
- CLIP is Almost All You Need: Towards Parameter-Efficient Scene Text Retrieval without OCR (CVPR 2025 · similarity 0.89)
- Revisiting Scene Text Recognition: A Data Perspective (ICCV 2023 · similarity 0.88)
- Turning a CLIP Model Into a Scene Text Spotter (TPAMI 2024 · similarity 0.88)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.88)
- Building Vision-Language Models on Solid Foundations with Masked Distillation (CVPR 2024 · similarity 0.87)
- SuperCLIP: CLIP with Simple Classification Supervision (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
