Research map / Vision-language and multimodal / Human and embodied modeling
Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition
Ayan Kumar Bhunia, Aneeshan Sain, Amandeep Kumar, Shuvozit Ghose, Pinaki Nath Chowdhury, Yi-Zhe Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Gumbel-softmax differentiable semantic space instead of logits (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 82.8 -> 84.0 | Improved |
| add Gumbel-softmax instead of straight-through estimator (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 80.1 -> 84.0 | Improved |
| add refinement stages beyond a single stage (s=1 to s=2) (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 80.3 -> 84.0 | Improved |
| add a third refinement stage (s=3) (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 84.0 -> 83.6 | No clear effect |
| add multi-scale feature maps B_{L-s} per stage (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 81.3 -> 84.0 | Improved |
| use FPN lateral connections instead of pyramid pooling (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 81.9 -> 84.0 | Improved |
| add visual reasoning block (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 2.9% drop when removed | Improved |
| add semantic reasoning block (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 4.8% drop when removed | Improved |
| add dense connection between glimpse vectors (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 1.6% drop when removed | Improved |
| add residual connection between first and last stage (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 1.3% drop when removed | Improved |
| add multi-stage loss plus auxiliary visual and semantic losses over last-stage loss only (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 73.1 -> 84.0 | Improved |
| use Transformer instead of Bi-LSTM reasoning modules (ablation) | Proposed (Stage-2) · IC15 | word recognition accuracy: 81.8 -> 83.1 | Improved |
Closest papers by meaning
- Multi-modal Text Recognition Networks: Interactive Enhancements between Visual and Semantic Features (ECCV 2022 · similarity 0.91)
- Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition (AAAI 2022 · similarity 0.91)
- OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning (NeurIPS 2025 · similarity 0.89)
- Image as a Language: Revisiting Scene Text Recognition via Balanced, Unified and Synchronized Vision-Language Reasoning Network (AAAI 2024 · similarity 0.89)
- OTE: Exploring Accurate Scene Text Recognition Using One Token (CVPR 2024 · similarity 0.89)
- Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
