Research map / Architectures and efficiency / Attention and transformers
TAMER: Tree-Aware Transformer for Handwritten Mathematical Expression Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
TAMER adds a tree-aware module to a CoMER-style Transformer that jointly predicts LaTeX sequences and parent-child tree structure, plus a tree-scoring step in beam search, reaching SOTA ExpRate on CROHME 2014/2016/2019 without data augmentation.
Problem
Sequence-based decoders for handwritten mathematical expression recognition (HMER) predict LaTeX directly, so they under-model the tree structure of expressions and often produce syntactically invalid output such as unmatched braces.
Approach
Add a Tree-Aware Module on top of the Transformer decoder that predicts a parent-child score matrix over characters, train it jointly with the sequence loss, and at inference rescore beam-search candidates with the tree structure score, keeping parallel Transformer training.
Key results
- TAMER reaches 61.23% ExpRate on CROHME 2014, 60.26% on CROHME 2016 and 61.97% on CROHME 2019 without data augmentation, versus reproduced CoMER at 58.38%, 56.98% and 59.12%.
- On HME100K TAMER gets 68.52% ExpRate, and TAMER with the Fusion Module gets 69.50%, the latter the best reported in that table.
- Ablation on CROHME 2014 shows the Tree-Aware Module alone gives 60.39% and adding tree scoring gives 61.23%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the Tree-Aware Module (ablation) | CoMER · CROHME 2014/2016/2019 test | ExpRate: 58.38 -> 60.39; 56.98 -> 59.08; 59.12 -> 61.32 | Helped |
| add Tree Structure Prediction Scoring (ablation) | TAM · CROHME 2014/2016/2019 test | ExpRate: 60.39 -> 61.23; 59.08 -> 60.26; 61.32 -> 61.97 | Helped |
| add Fusion Module (ablation) | TAMER · HME100K | ExpRate: 68.52 -> 69.50 | Helped |
| add Tree-Aware Module and Tree Scoring (ablation) | CoMER · CROHME 2014/2016/2019 test | ExpRate: 58.38 -> 61.23; 56.98 -> 60.26; 59.12 -> 61.97 | Helped |
| use Tree Structure Prediction Scoring during inference (ablation) | TAMER · CROHME 2014 test | FPS: 9.45 -> 6.75 | Hurt |
| add the Tree-Aware Module | CoMER · CROHME 2014 test, structural complexity 5 or higher | ExpRate: 5.31% | Helped |
| add the Tree-Aware Module and tree-based joint training | CoMER · CROHME 2014/2016/2019 test | ExpRate: 2.85%; 3.28%; 2.85% | Helped |
| add the Tree-Aware Module | CoMER · CROHME 2014, structural complexity above 5 | bracket matching accuracy: more than 15% | Helped |
| add the Tree-Aware Module | CoMER · CROHME 2014/2016/2019 and HME100K, complex structures | bracket matching accuracy: 92.65%; 96.86%; 94.74%; 93% | Helped |
Surprising findings
- Tree scoring at inference cuts FPS from 9.45 to 6.75 on a 2080Ti, a notable speed cost for the gain.
Limitations
- Tree scoring slows inference substantially (FPS 11.13 to 6.75).
- Baseline CoMER and TAMER results are reproduced by the authors, and the CROHME comparisons exclude data augmentation, so numbers for prior methods are taken from their papers.
- Gains on HME100K are smaller, and TAMER without fusion slightly trails ICAL on ExpRate there.
Experimental setup
Datasets: CROHME 2014 CROHME 2016 CROHME 2019 HME100K
Benchmarks: CROHME 2014 CROHME 2016 CROHME 2019 HME100K
Metrics: ExpRate <=1 error <=2 errors bracket matching accuracy
Baselines: CoMER ICAL NAMER GCN CAN-ABM ABM SAN TDv2 DenseWAP-TD BTTR
Builds on: CoMER DenseNet Transformer BTTR ICAL
Closest papers by meaning
- TDv2: A Novel Tree-Structured Decoder for Offline Mathematical Expression Recognition (AAAI 2022 · similarity 0.92)
- Syntax-Aware Network for Handwritten Mathematical Expression Recognition (CVPR 2022 · similarity 0.91)
- Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition (NeurIPS 2025 · similarity 0.89)
- SSAN: A Symbol Spatial-Aware Network for Handwritten Mathematical Expression Recognition (AAAI 2025 · similarity 0.87)
- Generating Handwritten Mathematical Expressions From Symbol Graphs: An End-to-End Pipeline (CVPR 2024 · similarity 0.85)
- Separations in the Representational Capabilities of Transformers and Recurrent Architectures (NeurIPS 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.