Research map / Label-efficient and robust learning / Recognition and long-tail
Atom-Level Optical Chemical Structure Recognition with Limited Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AtomLenz, an atom-level object-detection-based optical chemical structure recognizer pretrained on synthetic images and adapted to hand-drawn molecules using only SMILES via ProbKT* and edit-correction, plus a ChemExpert combination with DECIMER.
Problem
Optical chemical structure recognition (OCSR) generalizes poorly to data-scarce domains like hand-drawn molecule images, and atom-level localization methods require costly atom-level annotations that are unavailable in such domains.
Approach
Use a Faster R-CNN atom-level entity detector pretrained on synthetic images, then weakly supervised domain adaptation with only SMILES via ProbKT* (probabilistic reasoning with Hungarian matching and relabeling) combined with a graph edit-correction mechanism that generates pseudo atom-level labels; a ChemExpert module chains it with DECIMER for chemically valid predictions.
Key results
- On the hand-drawn test set, ChemExpert (DECIMER fine-tuned + AtomLenz+EditKT*) reaches 0.635 accuracy and 0.749 Tanimoto, versus 0.622 / 0.727 for DECIMER fine-tuned alone.
- AtomLenz+EditKT* improves over non-fine-tuned AtomLenz from 0.009 to 0.338 accuracy on the hand-drawn test set.
- When retrained from scratch on 4,070 hand-drawn samples, AtomLenz reaches 0.338 accuracy versus 0.0 to 0.013 for Img2Mol, MolScribe, ChemGrapher and DECIMER.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune AtomLenz using EditKT* | AtomLenz · Hand-drawn test set | accuracy (T=1): 0.009 -> 0.338 | Helped |
| Fine-tune AtomLenz using EditKT* | AtomLenz · Hand-drawn test set | Tanimoto similarity: 0.087 -> 0.484 | Helped |
| Fine-tune AtomLenz using EditKT* | AtomLenz · ChemPix test set | accuracy (T=1): 0.054 -> 0.484 | Helped |
| Fine-tune AtomLenz using EditKT* | AtomLenz · ChemPix test set | Tanimoto similarity: 0.064 -> 0.605 | Helped |
| Combine DECIMER fine-tuned with AtomLenz+EditKT* | AtomLenz+EditKT* · Hand-drawn test set | accuracy (T=1): 0.338 -> 0.635 | Helped |
| Combine DECIMER fine-tuned with AtomLenz+EditKT* | AtomLenz+EditKT* · Hand-drawn test set | Tanimoto similarity: 0.484 -> 0.749 | Helped |
| Combine DECIMER fine-tuned with AtomLenz+EditKT* | AtomLenz+EditKT* · ChemPix test set | accuracy (T=1): 0.484 -> 0.518 | Helped |
| Combine DECIMER fine-tuned with AtomLenz+EditKT* | AtomLenz+EditKT* · ChemPix test set | Tanimoto similarity: 0.605 -> 0.655 | Helped |
| Use AtomLenz instead of the other methods when retrained from scratch on the same 4,070-sample hand-drawn training set | Chemical structure recognition models · Hand-drawn test set (4,070-sample training) | accuracy (T=1): 0.013 -> 0.338 | Helped |
| Use AtomLenz instead of the other methods when retrained from scratch on the same 4,070-sample hand-drawn training set | Chemical structure recognition models · Hand-drawn test set (4,070-sample training) | Tanimoto similarity: 0.0865 -> 0.484 | Helped |
Surprising findings
- DECIMER, trained on about 2000 times more images, is outperformed by the fine-tuned AtomLenz on atom-level localization only in mAP terms; DECIMER gets higher count accuracy (0.973) despite no localization.
- Retrained from scratch on the small hand-drawn set, AtomLenz is far more data efficient than baselines, which mostly collapse to near-zero accuracy.
Limitations
- Performance on hand-drawn images remains far from reliable; the best single model reaches only 0.338 accuracy after adaptation.
- Fails on rare atom or bond types, which the edit-correction mechanism only partly addresses.
- Atom localization mAP is low (0.801 count accuracy, but mAP only reported as 0.801 vs 0.008 for MolScribe on a different metric scale), and the ChemExpert gain relies on combining with an external tool (DECIMER).
Experimental setup
Datasets: ChEMBL Hand-drawn dataset (Brinkhaus et al.) ChemPix (Weir et al.) Atom localization dataset (Oldenhof et al.)
Benchmarks: hand-drawn test set ChemPix test set atom localization test set
Metrics: accuracy (ECFP6 Tanimoto=1) average Tanimoto similarity count accuracy mAP
Baselines: DECIMER v2.2.0 DECIMER fine-tuned Img2Mol MolScribe ChemGrapher OSRA
Builds on: ProbKT Faster R-CNN RDKit DECIMER
Closest papers by meaning
- RFL: Simplifying Chemical Structure Recognition with Ring-Free Language (AAAI 2025 · similarity 0.89)
- MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures (CVPR 2025 · similarity 0.89)
- Atomas: Hierarchical Adaptive Alignment on Molecule-Text for Unified Molecule Understanding and Generation (ICLR 2025 · similarity 0.86)
- Learning Over Molecular Conformer Ensembles: Datasets and Benchmarks (ICLR 2024 · similarity 0.85)
- TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks (AAAI 2026 · similarity 0.85)
- GLACIER: Rethinking Mass Spectrum Prediction as an Object Detection Problem (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.