Research map / Vision-language and multimodal / Human and embodied modeling
Enhanced Optical Character Recognition by Optical Sensor Combined with BERT and Cosine Similarity Scoring (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BERTOCR, which routes low-confidence OCR outputs on noisy text images to BERT masked-token prediction and re-ranks candidates by cosine similarity of rendered word images, reporting about 4.3x the accuracy of conventional OCR.
Problem
OCR accuracy degrades on noisy text images, and conventional OCR ignores sentence context.
Approach
Uses OCR confidence with threshold K=0.5 to decide whether to trust the OCR output; for low-confidence words, BERT proposes masked-token candidates and a score combining BERT probability with image cosine similarity between rendered candidates and the noisy word selects the final output.
Key results
- Conventional OCR averages 0.0594 accuracy across the three datasets, versus 0.2486 for OCR and BERT and 0.2576 for BERTOCR.
- BERTOCR outperforms conventional OCR by a factor of 4.3367 on average accuracy.
- BERTOCR is 1.0362x (3.6%) better than OCR combined with BERT alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BERT and cosine similarity scoring to conventional OCR | conventional OCR · Average | accuracy: 0.0594 -> 0.2576 | Helped |
| Add cosine similarity scoring to OCR and BERT (ablation) | OCR and BERT · Average | accuracy: 0.2486 -> 0.2576 | Helped |
| Add cosine similarity scoring to OCR and BERT (ablation) | OCR and BERT · Wiki | accuracy: 0.3641 -> 0.3719 | Helped |
| Add cosine similarity scoring to OCR and BERT (ablation) | OCR and BERT · CNN | accuracy: 0.1979 -> 0.2086 | Helped |
| Add cosine similarity scoring to OCR and BERT (ablation) | OCR and BERT · Fiction | accuracy: 0.1838 -> 0.1923 | Helped |
Limitations
- Evaluated only on synthetic gray-line noise injected into text images.
- Absolute accuracy remains low (about 0.26 at best).
- OCR engine used is not named and the threshold K=0.5 is fixed without sensitivity analysis.
Experimental setup
Datasets: Wikipedia sentences CNN/DailyMail sentences Fiction sentences
Benchmarks: Wikipedia (noisy-injected) CNN/DailyMail (noisy-injected) Fiction (noisy-injected)
Metrics: accuracy
Baselines: conventional OCR OCR and BERT
Builds on: BERT
Closest papers by meaning
- UPOCR: Towards Unified Pixel-Level OCR Interface (ICML 2024 · similarity 0.84)
- ODM: A Text-Image Further Alignment Pre-training Approach for Scene Text Detection and Spotting (CVPR 2024 · similarity 0.83)
- OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning (NeurIPS 2025 · similarity 0.82)
- MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios (NeurIPS 2025 · similarity 0.82)
- Nougat: Neural Optical Understanding for Academic Documents (ICLR 2024 · similarity 0.81)
- Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR (CVPR 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.