Research map / Label-efficient and robust learning / OOD and anomaly detection
On the Generalization of Handwritten Text Recognition Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies out-of-distribution generalization of eight HTR architectures trained from scratch on seven handwriting datasets (336 OOD cases), uses synthetic data, and factor analysis to show textual divergence drives OOD error and can be used to predict it.
Problem
Handwritten text recognition models are evaluated under the i.i.d. assumption, and their ability to generalize to out-of-distribution handwriting (new manuscripts, languages, alphabets) without any target-domain access is unexplored.
Approach
Frames HTR generalization as single-source domain generalization with no target data access, and identifies latent factors (textual divergence, visual divergence) via factor analysis that explain OOD CER and allow its estimation from proxy metrics.
Key results
- Average OOD CER across eight HTR architectures ranges from 37.4% (VAN) to 53.9% (VLT), versus ID CERs around 5-10%.
- Textual divergence loads strongly on factor 1 (0.92 ground-truth, 0.84 generic text) and correlates 0.62 with OOD error; visual divergence correlates only -0.3 with OOD error.
- OOD CER can be estimated from label-free proxy metrics with MAE of 10.9 points; about 70% of predictions fall within 10 points error.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CTC alignment rather than autoregressive alignment | HTR models in the OOD setting · Seven datasets (average) | CER (%): 40.4 vs 48.7 | Helped |
| Train with synthetic data instead of real data | HTR models evaluated on real datasets · OOD settings | CER (%): 6 points | Helped |
| Train using synthetic data in the target language rather than a different language | HTR models evaluated on real datasets · OOD settings | CER (%): +11 CER points | Helped |
| Select the model using held-out domains or target-domain oracle results instead of source validation | HTR models · OOD settings | CER (%) | No clear effect |
| Increase model capacity | HTR models · OOD settings | CER (%) | No clear effect |
| Increase textual divergence between source and target domains | HTR models · OOD settings | CER (%): 0.62 | Hurt |
| Increase visual divergence between source and target domains | HTR models · OOD settings | CER (%): -0.3 | Hurt |
| Use CRNN rather than the other evaluated architectures | HTR models trained with synthetic data · Real historical datasets | CER (%) | Helped |
| Increase model capacity | HTR models · OOD calibration | ECE | No clear effect |
Surprising findings
- Model selection strategy (ID, held-out, oracle) has negligible impact on OOD error, contrary to broader domain generalization literature.
- Larger model capacity does not improve OOD generalization.
Limitations
- Visual divergence is measured with a simple convolutional autoencoder and contributes only weakly, so its role is not fully resolved.
- OOD errors remain high (37-54% CER) for all architectures, and the error estimator has an average deviation of 10.9 points.
Experimental setup
Datasets: IAM Rimes Bentham Saint-Gall George Washington Rodrigo ICFHR 2016 WIT
Benchmarks: IAM Rimes Bentham Saint-Gall George Washington Rodrigo ICFHR 2016
Metrics: Character Error Rate Expected Calibration Error Mean Absolute Error Pearson correlation
Baselines: CRNN VAN C-SAN HTR-VT Kang Michael LT VLT
Builds on: Connectionist Temporal Classification Domain Generalization literature (model selection, held-out and oracle selection
Closest papers by meaning
- MetaWriter: Personalized Handwritten Text Recognition Using Meta-Learned Prompt Tuning (CVPR 2025 · similarity 0.87)
- General Detection-based Text Line Recognition (NeurIPS 2024 · similarity 0.86)
- CLIPCEIL: Domain Generalization through CLIP via Channel rEfinement and Image-text aLignment (NeurIPS 2024 · similarity 0.85)
- PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks (CVPR 2026 · similarity 0.85)
- Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization (ICLR 2024 · similarity 0.85)
- Out of Length Text Recognition with Sub-String Matching (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.