Research map / Label-efficient and robust learning / Recognition and long-tail
SSAN: A Symbol Spatial-Aware Network for Handwritten Mathematical Expression Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSAN, an auxiliary task and network that predicts 2D Gaussian symbol spatial distribution maps (derived from printed LaTeX templates, then refined to decoder attention maps) jointly with attention-based HMER models, improving CROHME and HME100K recognition.
Problem
Handwritten mathematical expression recognition struggles with complex 2D symbol spatial structures; existing attention, counting, and tree-based methods do not effectively capture symbol spatial information, causing attention drift and symbol misclassification.
Approach
Use a multi-task auxiliary task predicting a symbol spatial distribution map (Gaussian maps centered on connected components of rendered printed LaTeX templates), with a coarse-to-fine target switch to the trained decoder's attention maps and attention-guided symbol masking to separate similar symbols.
Key results
- DWAP+SSAN improves over DWAP by 9.37% (CROHME 2014), 9.37% (CROHME 2016), 11.79% (CROHME 2019) and 6.74% (HME100K) ExpRate.
- CAN-DWAP+SSAN leads on CROHME 2016, 2019 and HME100K; DWAP+SSAN achieves state-of-the-art on CROHME 2014 among compared methods (60.85% ExpRate).
- SSAN outperforms CAN and SAM auxiliary tasks when added to DWAP (60.85 vs 57.00 and 56.80 on CROHME 2014); combining all three auxiliary tasks does not beat SSAN alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SSAN to DWAP | DWAP · CROHME 2014 | ExpRate: 51.48 -> 60.85; 9.37% | Helped |
| Add SSAN to DWAP | DWAP · CROHME 2016 | ExpRate: 50.65 -> 60.02; 9.37% | Helped |
| Add SSAN to DWAP | DWAP · CROHME 2019 | ExpRate: 50.04 -> 61.83; 11.79% | Helped |
| Add SSAN to DWAP | DWAP · HME100K | ExpRate: 61.85 -> 68.74; 6.74% | Helped |
| Add SSAN joint optimization without distribution loss (ablation) | DWAP · CROHME 2014 | ExpRate: 51.48 -> 56.69 | Helped |
| Add symbol distribution loss (ablation) | DWAP with SSAN joint optimization · CROHME 2014 | ExpRate: 56.69 -> 58.72 | Helped |
| Add coarse-to-fine alignment (ablation) | DWAP with distribution loss · CROHME 2014 | ExpRate: 58.72 -> 59.53 | Helped |
| Add attention-guided symbol masking (ablation) | DWAP with coarse-to-fine alignment · CROHME 2014 | ExpRate: 59.53 -> 60.85 | Helped |
| Combine SSAN with CAN (ablation) | DWAP + SSAN · CROHME 2014 | ExpRate: 60.85 -> 59.03 | Hurt |
| Combine SSAN with SAM (ablation) | DWAP + SSAN · CROHME 2019 | ExpRate: 61.83 -> 60.63 | Hurt |
| Combine SSAN, CAN, and SAM (ablation) | DWAP · CROHME 2019 | ExpRate: 61.83 -> 61.05 | Hurt |
Surprising findings
- Combining SSAN with CAN or SAM auxiliary tasks gives little gain or even lowers performance versus SSAN alone, attributed to inconsistent learning targets and higher optimization complexity.
- DWAP+SSAN has 3.5x the parameters of DWAP but is only about 6.5% slower in FPS.
Limitations
- Symbol spatial targets from printed templates only loosely align with handwritten symbol positions, motivating the coarse-to-fine stage.
- Gains from combining multiple auxiliary tasks are not additive, and combined tasks can hurt performance.
- Experiments use DWAP-based models mainly; generality to other HMER architectures is shown only with CAN and SAM.
Experimental setup
Datasets: CROHME 2014 CROHME 2016 CROHME 2019 HME100K
Benchmarks: CROHME 2014 CROHME 2016 CROHME 2019 HME100K
Metrics: ExpRate <=1 error <=2 error
Baselines: WAP DWAP-MSA WS-WAP PAL-v2 DWAP-TD BTTR TDv2 TSDNet ABM SAN CoMER SLAN
Builds on: DWAP CAN SAM TDAM
Closest papers by meaning
- Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition (NeurIPS 2025 · similarity 0.90)
- Generating Handwritten Mathematical Expressions From Symbol Graphs: An End-to-End Pipeline (CVPR 2024 · similarity 0.90)
- TAMER: Tree-Aware Transformer for Handwritten Mathematical Expression Recognition (AAAI 2025 · similarity 0.87)
- SpaceGTN: A Time-Agnostic Graph Transformer Network for Handwritten Diagram Recognition and Segmentation (AAAI 2024 · similarity 0.82)
- A Neural-Guided Dynamic Symbolic Network for Exploring Mathematical Expressions from Data (ICML 2024 · similarity 0.81)
- Symbol as Points: Panoptic Symbol Spotting via Point-based Representation (ICLR 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.