Research map / Vision-language and multimodal / Human and embodied modeling
RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioning
Jiahe Song, Chuang Wang, Bowen Jiang, Yinfan Wang, Hao Zheng, Xingjian Wei, Chengjin Liu, Rui Nie, Junyuan Gao, Jiaxing Sun, Yubin Wang, Lijun Wu and 4 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BIVP (box+index visual prompt) instead of BROS coordinate output (ablation) | RxnCaption-VL · RxnScribe-test | Soft F1: 76.2 -> 86.2 | Improved |
| use BIVP instead of BROS (ablation) | RxnCaption-VL · U-RxnDiagram-15k-test | Hybrid F1: 57.2 -> 59.8 | Improved |
| use BIVP instead of BROS (ablation) | Gemini-2.5-Pro (zero-shot) · RxnScribe-test | Soft F1: 24.3 -> 76.1 | Improved |
| replace YoDe with MolYOLO detector (ablation) | RxnCaption-VL · RxnScribe-test | Hybrid F1: 53.3 -> 72.2 | Improved |
| replace MolDetect with MolYOLO detector (ablation) | RxnCaption-VL · RxnScribe-test | Hybrid F1: 70.8 -> 72.2 | Improved |
| supply ground-truth boxes to BIVP detector (ablation) | RxnCaption-VL · U-RxnDiagram-15k-test | Hybrid F1: 59.8 -> 63.8 | Improved |
| use BIVP instead of BROS (inference speed) (ablation) | RxnCaption-VL · A100 speed | seconds per image: 16.88 -> 9.65 | Improved |
| train RxnScribe on U-RxnDiagram-15k (ablation) | RxnScribe · U-RxnDiagram-15k-test | Hybrid F1: 34.9 -> 47.4 | Improved |
| train RxnScribe on U-RxnDiagram-15k (ablation) | RxnScribe · RxnScribe-test | Hybrid F1: 69.1 -> 70.7 | Improved |
| use large synthetic RxnIM data instead of real data (ablation) | RxnIM · U-RxnDiagram-15k-test | Soft F1: 45.9 -> 40.5 | Hurt |
Closest papers by meaning
- ReactGPT: Understanding of Chemical Reactions via In-Context Tuning (AAAI 2025 · similarity 0.90)
- TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks (AAAI 2026 · similarity 0.89)
- Reaction Graph: Towards Reaction-Level Modeling for Chemical Reactions with 3D Structures (ICML 2025 · similarity 0.87)
- ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area (AAAI 2025 · similarity 0.86)
- MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures (CVPR 2025 · similarity 0.85)
- Reaction Prediction via Interaction Modeling of Symmetric Difference Shingle Sets (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
