Research map / Video / Video-language understanding
GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Turns a frozen query-based image text spotter (DeepSolo) into a video text spotter by adding a lightweight rescoring head and a long-short term Transformer matcher (LST-Matcher), tuned on video data, and introduces the ArTVideo curved-text test set.
Problem
Video text spotting lags far behind image text spotting in recognition quality, and jointly training detection, recognition and tracking end-to-end causes optimization conflicts and high training cost.
Approach
Freeze an off-the-shelf RoI-free image text spotter (DeepSolo), recalibrate its query confidences with a trainable rescoring head fused by max, and associate query-based instances across frames with a two-stage short-term and long-term Transformer matcher, so training focuses only on tracking.
Key results
- On ICDAR15-video, GoMatching (size 1000) reaches 72.04 MOTA, 78.53 MOTP and 80.11 IDF1, improving over TransDETR's 60.96 MOTA by over 11 points.
- On BOVText, GoMatching reports 52.9 MOTA versus 11.4 for CoText, the previous best.
- On DSText, GoMatching (single model) reaches 22.83 MOTA, ranking first among the leaderboard entries, and 46.09 IDF1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace RoI features with frozen DeepSolo queries for association (ablation) | GoMatching with LT-Matcher on ICDAR15-video · ICDAR15-video | MOTA: 66.20 -> 67.22 | Helped |
| use rescoring-head scores instead of original DeepSolo scores (ablation) | GoMatching with LT-Matcher on ICDAR15-video · ICDAR15-video | MOTA: 67.22 -> 68.47 | Helped |
| fuse original and rescoring-head scores using maximum (ablation) | GoMatching with LT-Matcher on ICDAR15-video · ICDAR15-video | MOTA: 68.47 -> 68.80 | Helped |
| add ST-Matcher to LT-Matcher (ablation) | GoMatching on ICDAR15-video · ICDAR15-video | MOTA: 68.80 -> 70.52 | Helped |
| add LT-Matcher to ST-Matcher (ablation) | GoMatching on ICDAR15-video · ICDAR15-video | IDF1: +5.1% | Helped |
| fine-tune the image spotter before tracker training rather than keeping it frozen (ablation) | GoMatching on ICDAR15-video · ICDAR15-video | MOTA: 72.04 -> 70.82 | Hurt |
| jointly train the spotter decoder and tracker at decoder LR 0.001 (ablation) | GoMatching on ICDAR15-video · ICDAR15-video | IDF1: 80.11 -> 78.98 | Hurt |
| increase joint-training decoder LR from 0.001 to 0.1 (ablation) | GoMatching on ICDAR15-video · ICDAR15-video | MOTA: 71.48 -> 68.03 | Hurt |
| use 6 rather than 32 LT-Matcher frames (ablation) | GoMatching on ICDAR15-video · ICDAR15-video | MOTA: 70.13 -> 70.52 | Helped |
| use maximum rather than mean score fusion (ablation) | GoMatching on ICDAR15-video · ICDAR15-video | MOTA: 70.46 -> 70.52 | Helped |
| increase inference image size from 800 to 1000 (ablation) | GoMatching on ICDAR15-video · ICDAR15-video | MOTA: 68.51 -> 72.04 | Helped |
| add the rescoring mechanism (ablation) | GoMatching on ICDAR13-video · ICDAR13-video | F-measure: 76.8 -> 81.5 | Helped |
Surprising findings
- Jointly fine-tuning the image spotter's decoder with the tracker degrades performance, even at a near-zero decoder learning rate.
- Fine-tuning the image text spotter on video frames before tracking also lowers results versus keeping it frozen.
Limitations
- Fails on very small text and extreme motion blur, inheriting the frozen image spotter's limits.
- Joint end-to-end optimization of spotter and tracker degrades results, so the approach avoids it rather than solving it.
- Performance depends on the quality of the off-the-shelf image spotter; the paper notes stronger spotters would help.
Experimental setup
Datasets: ICDAR15-video DSText BOVText ICDAR13-video CTW1500
Benchmarks: ICDAR15-video BOVText DSText ArTVideo ICDAR13-video
Metrics: MOTA MOTP IDF1 F-measure AP
Baselines: TransDETR CoText HIK_OCR TransVTSpotter TencentOCR DA DeepSolo
Builds on: DeepSolo MOTR GTR DETR focal loss
Closest papers by meaning
- SemiETS: Integrating Spatial and Content Consistencies for Semi-Supervised End-to-end Text Spotting (CVPR 2025 · similarity 0.88)
- Bridging the Gap Between End-to-End and Two-Step Text Spotting (CVPR 2024 · similarity 0.88)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.85)
- InstructOCR: Instruction Boosting Scene Text Spotting (AAAI 2025 · similarity 0.84)
- ODM: A Text-Image Further Alignment Pre-training Approach for Scene Text Detection and Spotting (CVPR 2024 · similarity 0.84)
- OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.