AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

CLIPTER: Looking at the Bigger Picture in Scene Text Recognition

Aviad Aberdam, David Bensaid, Alona Golts, Roy Ganz, Oren Nuriel, Royee Tichauer, Shai Mazor, Ron Litman

ICCV 2023 · Vision-language and multimodal › CLIP and zero-shot · approach: Fine-grained improvements to CLIP · 38 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add CLIPTER scene-level CLIP features via gated cross-attentionPARSeq · 12 scene text benchmarksweighted average word accuracy: +0.8%Improved
add CLIPTER to ABINet-VisABINet-Vis · 12 scene text benchmarksweighted average word accuracy: +1.7%Improved
add CLIPTER on out-of-vocabulary wordsPARSeq · OOV benchmarkword accuracy (%): 68.97 -> 71.45Improved
train with CLIPTER at 10% of data versus baseline at 25%PARSeq · low-data regimeword accuracyImproved
add CLIPTER to GLASS+PARSeq two-stage spotterGLASS + PARSeq · ICDAR 2015 E2EE2E score: 77.3 -> 77.4Improved
use vision-language image encoder (CLIP/BLIP/GIT) instead of vision-only (DINO/ViT-MAE/OWL-ViT) (ablation)PARSeq CLIPTER · 12 scene text benchmarksaverage word accuracy gain: +0.44-0.55 -> +0.53-0.73Improved
pool scene features to a single vector (k=inf) for BLIP in TRBA (ablation)TRBA CLIPTER · 12 scene text benchmarksaverage word accuracy gain: +0.60 (k=10) -> +0.15 (k=inf)Hurt
vary integration point (vision, contextual, decoder) (ablation)CLIPTER · 12 scene text benchmarksaverage word accuracy gain: ABINet Decoder +0.49 vs Contextual +0.18; ABINet-Vis Vision +1.73 vs Contextual +1.14Mixed
use more complex multi-head cross-attention fusion instead of gated attention (ablation)CLIPTER on PARSeq · 12 scene text benchmarksaverage word accuracy gain: +0.53 (gated) vs +0.55 (MH-CA mini)No clear effect
scale fusion module from MH-CA mini to small on TRBA (ablation)CLIPTER on TRBA · 12 scene text benchmarksaverage word accuracy gain: +0.67 -> +0.54Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy