Research map / Detection and segmentation / CNN features
SRFormer: Text Detection Transformer with Incorporated Segmentation and Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SRFormer, a DETR-based scene text detector that uses segmentation in only the first decoder layers as a regression prior and a Mask-informed Query Enhancement module, achieving SOTA on Total-Text, CTW1500 and ICDAR19-ArT without post-processing.
Problem
Scene text detection trades off segmentation-based methods (robust but needing costly CPU post-processing) against regression-based methods (simple post-processing but less robust and data-efficient); the paper seeks to combine both in one model.
Approach
Integrates segmentation into only the first few decoder layers of a DETR model to provide anchor points for progressive polygon-control-point regression in later layers, and uses the predicted masks as soft ROIs to pool pixel features that enhance instance queries (MQE).
Key results
- SRFormer outperforms DPText by +1.3% F1 on Total-Text (90.0 vs 88.7 with 2 seg layers), +0.7% on CTW1500 and +1.2% on ICDAR19-ArT.
- With only two decoder layers, SRFormer reaches 88.95% F1 on Total-Text versus 88.72% for the six-layer DPText, and it is 26.96 points ahead at layer 0 when trained from scratch.
- With 10% of Total-Text training data, SRFormer achieves 76.9% F1 versus 75.6% for DPText; on MLT17 cross-domain evaluation it gains +13.18%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SRFormer instead of DPText | Text detection · Total-Text | F1: 88.7 -> 90.0 | Helped |
| Use SRFormer instead of DPText | Text detection · CTW1500 | F1: 88.8 -> 89.6 | Helped |
| Use SRFormer instead of DPText | Text detection · ICDAR19-ArT | F1: 78.1 -> 79.3 | Helped |
| Use 3 segmentation and 3 regression decoder layers instead of 1 segmentation and 5 regression layers (ablation) | SRFormer · TotalText (no pre-training) | F1: 86.5 -> 87.1 | Helped |
| Use 3 segmentation and 3 regression decoder layers instead of 2 segmentation and 4 regression layers (ablation) | SRFormer · TotalText (no pre-training) | F1: 87.0 -> 87.1 | Helped |
| Add mask-generated anchor-point regression (ablation) | SRFormer (#3Seg) · TotalText (no pre-training) | F1: 85.5 -> 86.0 | Helped |
| Add MQE (ablation) | SRFormer (#3Seg) · TotalText (no pre-training) | F1: 85.5 -> 86.7 | Helped |
| Add both mask-generated anchor-point regression and MQE (ablation) | SRFormer (#3Seg) · TotalText (no pre-training) | F1: 85.5 -> 87.1 | Helped |
Surprising findings
- The first decoder layer already yields segmentation results that barely improve in deeper layers, so adding more segmentation layers gives only marginal gains.
- Two decoder layers of SRFormer outperform the full six-layer DPText in the pretrained setting.
Limitations
- Adding MQE reduces inference speed (FPS drops from 10.5 to 7.9) and adds parameters.
- Results are shown on text-detection benchmarks only; generalization to other detection tasks is untested.
- Dice-based mask supervision is used only in the segmentation chunk, and the gains from additional segmentation layers are marginal.
Experimental setup
Datasets: SynthText150k MLT17 TotalText CTW1500 ICDAR19-ArT LSVT
Benchmarks: Total-Text Rot.Total-Text CTW1500 ICDAR19-ArT MLT17
Metrics: precision recall F1-score FPS
Baselines: DPText TESTR I3CL DBNet CRAFT PAN TextSnake ABCNet v2 FSG SwinTextSpotter TextBPN++
Builds on: DETR Deformable DETR DPText MaskDINO TESTR
Closest papers by meaning
- LayoutFormer: Hierarchical Text Detection Towards Scene Text Understanding (CVPR 2024 · similarity 0.85)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.85)
- ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining (AAAI 2024 · similarity 0.84)
- Text Grouping Adapter: Adapting Pre-trained Text Detector for Layout Analysis (CVPR 2024 · similarity 0.84)
- LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network (AAAI 2024 · similarity 0.84)
- Exploring Simple Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.