Research map / Large language models / Language, safety and interpretability

SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervision

ICLR 2025 · Large language models › Language, safety and interpretability · approach: Controlled text generation · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SMI-Editor, an edit-based pre-trained SMILES language model that is trained to restore valid molecules from fragment-dropped inputs via LevT-style deletion and insertion with a dual deletion loss.

Problem

Pre-trained SMILES masked language models saturate quickly during pre-training, capture substructure semantics poorly, and suffer a train-inference mismatch because they only see corrupted inputs with [MASK] tokens.

Approach

Randomly drop molecular fragments (BRICS/RMCF-based), feed the resulting valid SMILES to an encoder trained with Levenshtein Transformer-style deletion, placeholder insertion, and token prediction heads plus an added dual deletion loss, so the model learns to edit fragments back into the original molecule.

Key results

• SMI-Editor reaches a mean ROC-AUC of 77.8 over 7 MoleculeNet classification tasks under scaffold split, the best average in its table, beating MolCLR (75.5) and MoleculeSDE (74.5), and SMI-MLM (73.2).
• Under DeepChem splits, SMI-Editor achieves a mean of 85.2 versus 81.5 for SMI-MLM and 81.9 for DMP_TF.
• Replacing fragment dropping with random atom dropping lowers the mean ROC-AUC from 73.0 to 71.1 in the ablation on five tasks.

Measured effects reported in the paper

Surprising findings

• Increasing the MLM masking ratio to 45% did not alleviate rapid training-loss saturation.
• Ablation shows removing placeholder insertion causes the largest drop, larger than removing token deletion.

Limitations

• Main results use a different data split from the DeepChem split, with the DeepChem re-evaluation added later in the appendix.
• Evaluation limited to MoleculeNet classification tasks; no generation or regression benchmarks reported in main text.
• Fragment drop ratio and fragmentation rules are fixed heuristics (BRICS/RMCF) without broader sensitivity analysis.

Experimental setup

Datasets: 19M-molecule SMILES pre-training corpus MoleculeNet

Benchmarks: BACE BBBP Tox21 SIDER MUV ClinTox ToxCast ESOL FreeSolv HIV

Metrics: ROC-AUC mean ROC-AUC

Baselines: D-MPNN AttentiveFP N-Gram GROVER GraphMVP MolCLR Mole-BERT MoleculeSDE SMI-MLM

Builds on: Levenshtein Transformer BRICS RMCF MoleculeNet

Closest papers by meaning

• Text-Guided Molecule Generation with Diffusion Language Model (AAAI 2024 · similarity 0.86)
• Omni-Mol: Multitask Molecular Model for Any-to-any Modalities (NeurIPS 2025 · similarity 0.86)
• SALSA: Semantically-Aware Latent Space Autoencoder (AAAI 2024 · similarity 0.86)
• Atomas: Hierarchical Adaptive Alignment on Molecule-Text for Unified Molecule Understanding and Generation (ICLR 2025 · similarity 0.86)
• Improving Large Molecular Language Model via Relation-aware Multimodal Collaboration (AAAI 2026 · similarity 0.85)
• LICO: Large Language Models for In-Context Molecular Optimization (ICLR 2025 · similarity 0.85)