Research map / Vision-language and multimodal / Multimodal LLMs
Molecular Identifier Visual Prompting and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing
Jiahe Song, Chuang Wang, Yinfan Wang, Hao Zheng, Bowen Jiang, Rui Nie, Xingjian Wei, Junyuan Gao, Yubin Wang, Bin Wang, Lijun Wu, Jiang Wu and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use molecule identifiers (IdtVP) instead of overlaid indexed boxes (BIVP) | Gemini 3 Pro zero-shot · RxnScribe-test Hybrid Match | F1: 45.27 -> 71.70 | Improved |
| use identifiers instead of raw-image BROS | Gemini 3 Pro zero-shot · RxnScribe-test Hybrid Match | F1: 3.90 -> 71.70 | Improved |
| IdtVP on small open models zero-shot | Qwen2.5-VL-7B · RxnCaption-15k-test Soft Match | F1: 19.16 (BIVP 15.6 vs IdtVP 19.2) | No clear effect |
| add Re3-DAPO RL after SFT | RxnID (IdtVP) · RxnCaption-15k-test Hybrid Match | F1: 61.2 -> 64.4 | Improved |
| add Re3-DAPO RL after SFT | RxnCaption (BIVP) · RxnCaption-15k-test Hybrid Match | F1: 59.8 -> 64.1 | Improved |
| add Re3-DAPO RL after SFT | RxnCaption (BIVP) · ScannedRxn Hybrid Match | F1: 51.0 -> 45.1 | Hurt |
| add Re3-DAPO RL after SFT | RxnID (IdtVP) · ScannedRxn Hybrid Match | F1: 54.5 -> 56.3 | Improved |
| add Re3-DAPO RL after SFT | RxnID (IdtVP) · RxnScribe-test Hybrid Match | F1: 74.6 -> 75.0 | No clear effect |
| use mixed Soft:Hybrid reward 1:1 rather than Soft-only (ablation) | IdtVP+RL · RxnCaption-15k-test Hybrid Match | F1: 62.8 -> 64.4 | Improved |
| use mixed reward rather than Hybrid-only (ablation) | IdtVP+RL · RxnCaption-15k-test Hybrid Match | F1: 64.4 vs 64.4 | No clear effect |
| use human ground-truth identifiers instead of Mid-Mapper detector (ablation) | IdtVP+RL · RxnCaption-15k-test Soft Match | F1: 74.5 -> 73.6 | Hurt |
| IdtVP vs BIVP serialization noise after SFT | SFT models · training-set image-level error rate | error rate %: 12.88 -> 4.66 | Improved |
Closest papers by meaning
- TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks (AAAI 2026 · similarity 0.92)
- ReactGPT: Understanding of Chemical Reactions via In-Context Tuning (AAAI 2025 · similarity 0.89)
- LLM-Augmented Chemical Synthesis and Design Decision Programs (ICML 2025 · similarity 0.87)
- ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area (AAAI 2025 · similarity 0.86)
- Reaction Graph: Towards Reaction-Level Modeling for Chemical Reactions with 3D Structures (ICML 2025 · similarity 0.86)
- Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
