Research map / Large language models / Agents, code and math
MolLangBench: A Comprehensive Benchmark for Language-Prompted Molecular Structure Recognition, Editing, and Generation
Feiyang Cai, Jiahui Bai, Tao Tang, Guijuan He, Joshua Luo, Tianyu Zhu, Srikanth Pilla, Gang Li, Ling Liu, Feng Luo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GPT-5 instead of o3 | LLM · MolLangBench structure recognition | average recognition accuracy: 0.877 -> 0.923 | Improved |
| use image input instead of SMILES text | o4-mini · MolLangBench structure recognition | recognition accuracy: 0.772 vs o3-mini-level language results | Hurt |
| require atom localization in addition to recognition | LLMs · MolLangBench structure recognition | accuracy: GPT-5 0.923 -> 0.862 | Hurt |
| use GPT-5 on bond stereo (E/Z) | LLMs · MolLangBench bond stereo | recognition accuracy: GPT-5 0.655; o3 0.480; GPT-4o 0.390 | Mixed |
| use SELFIES instead of SMILES (ablation) | LLMs · MolLangBench | accuracy | Hurt |
| describe molecular structure before predicting property (ablation) | GPT-4o, GPT-4.1 · BBBP, BACE | accuracy: improvement exceeds 5% | Improved |
| vary chemical terminology or instruction phrasing (ablation) | o3 · MolLangBench editing/generation | accuracy | No clear effect |
| use image generation model instead of text LLM for editing | GPT Image 1 · MolLangBench molecule editing | editing accuracy: 0.855 (GPT-5) vs 0.080 | Hurt |
| generate de novo instead of editing | GPT-5 · MolLangBench | SMILES validity / accuracy: editing 0.945/0.855 -> generation 0.690/0.430 | Hurt |
Closest papers by meaning
- Omni-Mol: Multitask Molecular Model for Any-to-any Modalities (NeurIPS 2025 · similarity 0.90)
- Domain-Agnostic Molecular Generation with Chemical Feedback (ICLR 2024 · similarity 0.90)
- MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text Prompts (AAAI 2024 · similarity 0.90)
- MolVision: Molecular Property Prediction with Vision Language Models (NeurIPS 2025 · similarity 0.89)
- Hierarchical Graph Tokenization for Molecule-Language Alignment (ICML 2025 · similarity 0.89)
- TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks (AAAI 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
