Research map / Detection and segmentation / Open-set and few-shot detection
Text Grouping Adapter: Adapting Pre-trained Text Detector for Layout Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Text Grouping Adapter (TGA), a module that plugs pre-trained scene text detectors and spotters into a layout analysis pipeline, grouping text instances into paragraphs via instance feature assembling and one-to-many group mask prediction, with frozen or fully fine-tuned detectors.
Problem
Scene text layout analysis (grouping detected words or lines into paragraphs) has lagged behind text detection, and prior unified models train from scratch on the small HierText layout dataset, not exploiting well-trained text detectors and large detection datasets.
Approach
A plug-in adapter that converts diverse text detector outputs into instance masks, assembles instance features from a pixel embedding map, and adds a group mask prediction task with one-to-many Hungarian assignment to learn global group context, then predicts a pairwise affinity matrix for grouping; it can be used with frozen or fine-tuned detectors and cascaded for word-to-line-to-paragraph group
Key results
- TGA with frozen KNet-R50 outperforms Unified Detector on test Paragraph PQ by 2.7% relative.
- TGA† with frozen MaskDINO-R50 reaches 60.05 test Paragraph PQ, a 12.0% relative improvement over Unified Detector (53.60 line-based PQ baseline, 52.84 paragraph PQ).
- Full fine-tuning of MaskDINO-R50 with TGA beats its frozen counterpart by 4.2 Paragraph PQ on HierText validation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add TGA to frozen DBNetPP-R50 | DBNetPP-R50 · HierText test, word-based paragraphs | Paragraph PQ: 40.33 -> 42.23 | Helped |
| Add enhanced TGA to frozen MaskDINO-R50 | MaskDINO-R50 · HierText test, line-based paragraphs | Paragraph PQ: 53.60 -> 60.05 | Helped |
| Fine-tune the text detector instead of freezing it | TGA + KNet-R50 · HierText val, line-based layout | Paragraph PQ: 49.83 -> 51.85 | Helped |
| Fine-tune the text detector instead of freezing it | TGA + MaskDINO-R50 · HierText val, line-based layout | Paragraph PQ: 54.95 -> 59.19 | Helped |
| Add Pixel Embedding Map (ablation) | TGA with GMP, full fine-tuning · HierText | Paragraph PQ: 54.55 -> 55.03 | Helped |
| Add GMP (ablation) | TGA with Pixel Embedding Map, full fine-tuning · HierText | Paragraph PQ: 49.99 -> 55.03 | Helped |
| Use group masks rather than line masks for GMP supervision (ablation) | TGA with frozen text detector · HierText | Paragraph PQ: 47.32 -> 54.27 | Helped |
| Remove Dice loss from GMP (ablation) | TGA · HierText paragraphs | PQ: 53.72 -> 35.68 | Hurt |
| Remove BCE loss from GMP (ablation) | TGA · HierText paragraphs | PQ: 53.72 -> 53.88 | No clear effect |
| Freeze the text detector instead of fine-tuning it | TGA + MaskDINO-R50 · CTW1500 line detection | Line AP: 41.46 -> 61.18 | Helped |
| Freeze the text detector instead of fine-tuning it | TGA + KNet-R50 · HierText val, line-based layout | Paragraph PQ: 51.85 -> 49.83 | Hurt |
Surprising findings
- Frozen-detector TGA loses about 23.5% relative text detection performance on CTW1500 under full fine-tuning, showing overfitting to the small layout dataset.
Limitations
- Word-based grouping is weaker on small-size texts and relies on a detector with worse instance-level performance.
- Full fine-tuning overfits and degrades generalized text detection performance.
Experimental setup
Datasets: HierText CTW1500 MSRA-TD500 IC19-LSVT
Benchmarks: HierText CTW1500 MSRA-TD500 IC19-LSVT
Metrics: Precision Recall F1 Panoptic Quality AP H-mean
Baselines: Unified Detector Google Cloud OCR API Max-DeepLab-Cluster
Builds on: Unified Detector DBNet++ DeepSolo KNet MaskDINO
Closest papers by meaning
- LayoutFormer: Hierarchical Text Detection Towards Scene Text Understanding (CVPR 2024 · similarity 0.90)
- Towards End-to-End Unified Scene Text Detection and Layout Analysis (CVPR 2022 · similarity 0.89)
- EAST: An Efficient and Accurate Scene Text Detector (CVPR 2017 · similarity 0.87)
- Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes (CVPR 2019 · similarity 0.87)
- Towards Unified Multi-granularity Text Detection with Interactive Attention (ICML 2024 · similarity 0.87)
- Kernel Adaptive Convolution for Scene Text Detection via Distance Map Prediction (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.