Research map / Vision-language and multimodal / Cross-modal learning
Multi-modal Text Recognition Networks: Interactive Enhancements between Visual and Semantic Features
Byeonghu Na, Yoonsik Kim, Sungrae Park
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-modal transformer feature enhancement to ABINet (ablation) | ABINet · Total (six benchmarks) | word accuracy (%): 92.6 -> 93.0 | Improved |
| add spatial encoding to semantic features (SES) (ablation) | ABINet + multi-modal transformer · Total (six benchmarks) | word accuracy (%): 93.0 -> 93.3 | Improved |
| add visual clue masking (ablation) | ABINet + multi-modal transformer + SES · Total (six benchmarks) | word accuracy (%): 93.3 -> 93.5 | Improved |
| add visual clue masking (ablation) | ABINet + multi-modal transformer + SES · IC15 (1811) | word accuracy (%): 86.9 -> 86.6 | Hurt |
| add visual clue masking (ablation) | ABINet + multi-modal transformer + SES · CUTE80 | word accuracy (%): 89.9 -> 93.5 | Improved |
| update both visual and semantic features instead of only one modality (ablation) | ABINet with SES · Total (six benchmarks) | word accuracy (%): 92.8 -> 93.3 | Improved |
| enhance only semantic features (uni-modal FE) (ablation) | ABINet with SES · Total (six benchmarks) | word accuracy (%): 92.7 -> 92.8 | No clear effect |
| enhance only visual features (uni-modal FE) (ablation) | ABINet with SES · Total (six benchmarks) | word accuracy (%): 92.7 -> 92.8 | No clear effect |
| replace ABINet with MATRN | ABINet (reproduced) · CUTE80 | word accuracy (%): 89.0 -> 93.5 | Improved |
| replace ABINet with MATRN | ABINet (reproduced) · SVT | word accuracy (%): 93.7 -> 95.0 | Improved |
| scale up ABINet visual model to match MATRN parameters (VM-Big) (ablation) | ABINet · Total (six benchmarks) | word accuracy (%): 92.6 -> 92.9 | Improved |
| scale up ABINet visual model to match MATRN speed (VM-Bigger) (ablation) | ABINet · Total (six benchmarks) | word accuracy (%): 92.6 -> 93.1 | Improved |
Closest papers by meaning
- Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition (ICCV 2021 · similarity 0.91)
- CDistNet: Perceiving Multi-domain Character Distance for Robust Text Recognition (IJCV 2023 · similarity 0.91)
- OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning (NeurIPS 2025 · similarity 0.88)
- Image as a Language: Revisiting Scene Text Recognition via Balanced, Unified and Synchronized Vision-Language Reasoning Network (AAAI 2024 · similarity 0.88)
- Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition (CVPR 2025 · similarity 0.87)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
