Research map / Image generation / Super-resolution
GlyphSR: A Simple Glyph-Aware Framework for Scene Text Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GlyphSR, a scene text image super-resolution framework that uses a training-free SAM-based glyph extraction module to build character-level glyph masks, supervises glyph segmentation and classification, and fuses glyph maps into SR features.
Problem
Scene text image super-resolution (STISR) methods depend heavily on text priors from trained recognizers, which can be unreliable and cause incorrect glyph restoration when semantic information is unreliable.
Approach
Extract character-level glyph masks without training by prompting SAM with anchor points derived from recognizer attention and glyph-attention maps, then use these masks as supervision (glyph perception) and as spatial guidance fused into intermediate SR features (glyph fusion), rather than relying only on text priors.
Key results
- GlyphSR reaches 62.1% average recognition accuracy with CRNN on TextZoom, versus 59.0% for PEAN (+3.1%).
- With MORAN, GlyphSR averages 68.5% vs. 66.1% for PEAN (+2.4%); with ASTER it averages 71.0% vs. 70.6% (+0.4%).
- Ablation: adding GFM after the 4th SRB gives the best CRNN accuracy (62.09%); the training-free GEM masks give better recognition than K-means foreground-background pseudo-labels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add glyph-aware GEM, GPM, and GFM to the TPGSR backbone | GlyphSR · TextZoom, CRNN average accuracy | accuracy: 58.68% -> 62.09% | Helped |
| add glyph-aware GEM, GPM, and GFM to the TPGSR backbone | GlyphSR · TextZoom, MORAN average accuracy | accuracy: +2.4% | Helped |
| add glyph-aware GEM, GPM, and GFM to the TPGSR backbone | GlyphSR · TextZoom, ASTER average accuracy | accuracy: +0.4% | Helped |
| add glyph-aware GEM, GPM, and GFM to the TPGSR backbone | GlyphSR · TextZoom, ASTER, easy and medium subsets | accuracy: 84.5% -> 84.7%; 71.4% -> 71.1% | Mixed |
| replace K-means foreground-background pseudo-labels with GEM glyph masks (ablation) | GlyphSR without GFM · TextZoom, CRNN average accuracy | accuracy: 59.80% -> 61.01% | Helped |
| add GFM to GlyphSR (ablation) | GlyphSR · TextZoom, CRNN average accuracy | accuracy: 61.01% -> 62.09% | Helped |
| add GFM to GlyphSR (ablation) | GlyphSR · TextZoom | PSNR: 21.33 -> 20.74 | Hurt |
| add line-level glyph segmentation supervision (ablation) | GPM · TextZoom, CRNN average accuracy | accuracy: 58.68% -> 60.30% | Helped |
| add character-level glyph segmentation supervision (ablation) | GPM · TextZoom, CRNN average accuracy | accuracy: 58.68% -> 60.34% | Helped |
| add character classification supervision (ablation) | GPM · TextZoom, CRNN average accuracy | accuracy: 58.68% -> 60.14% | Helped |
| remove line-level glyph segmentation supervision (ablation) | GPM with all tasks · TextZoom, CRNN average accuracy | accuracy: 61.01% -> 60.30% | Hurt |
| remove character-level glyph segmentation supervision (ablation) | GPM with all tasks · TextZoom, CRNN average accuracy | accuracy: 61.01% -> 60.74% | Hurt |
Surprising findings
- Without GFM, the model achieves higher PSNR (21.33) and SSIM (0.7819) than the full GlyphSR (20.74, 0.7591), so adding GFM improves recognition at a cost in image fidelity.
- Adding GFM at the 3rd SRB drops CRNN average accuracy to 56.44%, well below the model without GFM (61.01%).
Limitations
- SAM sometimes fills hollow regions of characters such as 'a' and 'e' in the glyph masks.
- SAM may struggle with small-scale text images.
- Gains with the ASTER recognizer are small (+0.4% average).
Experimental setup
Datasets: TextZoom
Benchmarks: TextZoom
Metrics: text recognition accuracy PSNR SSIM
Baselines: TG TATT C3-STISR DPMN TSAN LEMMA RTSRN TextDiff RGDiffSR PEAN TBSRN TPGSR
Builds on: TPGSR PEAN C3-STISR SIGA Segment Anything Model
Closest papers by meaning
- GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing (CVPR 2025 · similarity 0.90)
- VecGlypher: Unified Vector Glyph Generation with Language Models (CVPR 2026 · similarity 0.86)
- GlyphAnchor: Enhancing Visual Text Rendering via Position-Anchored Glyph Priors (NeurIPS 2026 · similarity 0.86)
- Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders (NeurIPS 2025 · similarity 0.84)
- CLIP is Almost All You Need: Towards Parameter-Efficient Scene Text Retrieval without OCR (CVPR 2025 · similarity 0.84)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.