Research map / Image generation / Text-to-image diffusion
ODM: A Text-Image Further Alignment Pre-training Approach for Scene Text Detection and Spotting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OCR-Text Destylization Modeling (ODM), a text-prompted pixel-level pre-training that reconstructs uniform-style binary glyph images from scene images, with a Text-Controller module and automatic font-based label generation, improving scene text detectors and spotters fine-tuned from SynthText pre-training.
Problem
Text-image alignment for OCR pre-training: existing MIM and MLM pre-training methods do not explicitly align text prompts with their OCR-Text regions in images, and pixel-level labels are costly to obtain.
Approach
Pre-train an image encoder (ResNet50) with a text encoder via cross-attention to reconstruct a destylized binary glyph image of only the prompted OCR-Text, using Drop-Text and Noise-Text prompt perturbations, and generate pixel-level labels from font files plus existing text and box annotations.
Key results
- PSENet + ODM (SynthText pre-training) reaches 85.94 Hmean on TotalText vs 78.3 for PSENet baseline, and 85.62 on CTW1500 vs 78.0.
- DBNet++ + ODM improves Hmean on TotalText from 81.02 to 85.03 and on CTW1500 from 79.28 to 81.97.
- ODM-pretrained PSENet trained on SynthText alone surpasses oCLIP pre-trained on 40M web images on CTW1500 (85.62 vs 83.5 Hmean).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train the detector backbone with ODM on SynthText | FCENet · ICDAR15, CTW1500, TotalText | Hmean: 85.28 -> 86.54; 83.1 -> 83.60; 81.34 -> 82.32 | Mixed |
| pre-train the detector backbone with ODM on SynthText | DBNet++ · ICDAR15, CTW1500, TotalText | Hmean: 86.24 -> 88.81; 79.28 -> 81.97; 81.02 -> 85.03 | Helped |
| pre-train the detector backbone with ODM on SynthText | PSENet · ICDAR15, CTW1500, TotalText | Hmean: 79.98 -> 86.90; 78.0 -> 85.62; 78.3 -> 85.94 | Helped |
| pre-train the spotting model with ODM on SynthText | ABCNet, DeepSolo, and SPTS · ICDAR15 and CTW1500 | spotting scores (D-H, E-S, E-W, E-G, E-N, E-F): + 2.34; + 2.86; + 1.89; + 1.92; + 1.64; + 2.3; + 1.31; + 1.01; + 1.3; + 1.9; + 2.0; + 1.71; + 0.89; + 1.13; + 0.27; + 1. | Helped |
| pre-train with ODM using LSVT pseudo-label images | PSENet and DBNet++ · LSVT | Hmean: 68.23 -> 69.78; 75.02 -> 76.48 | Helped |
| set selected text proportion to 0% (ablation) | ODM with PSENet · CTW1500 | Hmean: 78.0 -> 83.29 | Helped |
| set selected text proportion to 30% (ablation) | ODM with PSENet · CTW1500 | Hmean: 78.0 -> 83.48 | Helped |
| set selected text proportion to 50% (ablation) | ODM with PSENet · CTW1500 | Hmean: 78.0 -> 84.27 | Helped |
| set selected text proportion to 70% (ablation) | ODM with PSENet · CTW1500 | Hmean: 78.0 -> 83.82 | Helped |
| add the Text Encoder (ablation) | ODM with PSENet · TotalText | Hmean: 81.68 -> 84.66 | Helped |
| add Drop-Text (ablation) | ODM with PSENet · TotalText | Hmean: 84.66 -> 85.44 | Helped |
| add Noise-Text (ablation) | ODM with PSENet · TotalText | Hmean: 84.66 -> 85.24 | Helped |
Surprising findings
- Pre-training on SynthText alone with ODM surpasses oCLIP pre-trained on 40 million web images on CTW1500.
Limitations
- Gains on LSVT weakly supervised pre-training are modest (about 1.5 Hmean points).
- Generated pixel-level labels may not align precisely with original character positions due to spacing discrepancies.
- Proportion of Drop-Text and Noise-Text must be tuned; too large or too small proportions reduce gains.
Experimental setup
Datasets: SynthText ICDAR2015 CTW1500 TotalText LSVT
Benchmarks: ICDAR2015 CTW1500 TotalText LSVT
Metrics: precision recall Hmean end-to-end F-score
Baselines: STKM oCLIP PSENet DBNet++ FCENet ABCNet DeepSolo SPTS
Builds on: CLIP MaskFeat oCLIP FPN OCR-VQGAN
Closest papers by meaning
- InstructOCR: Instruction Boosting Scene Text Spotting (AAAI 2025 · similarity 0.87)
- OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning (NeurIPS 2025 · similarity 0.86)
- UPOCR: Towards Unified Pixel-Level OCR Interface (ICML 2024 · similarity 0.86)
- PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks (CVPR 2026 · similarity 0.85)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.85)
- Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.