Research map / Probabilistic and scientific modeling / Molecules and proteins
Data-Efficient Molecular Generation with Hierarchical Textual Inversion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HI-Mol, a data-efficient molecular generator that applies hierarchical (shared, cluster-level, and molecule-level) textual inversion to the MolT5 Caption2SMILES model and samples by interpolating learned token embeddings.
Problem
Molecular generation when only a small number of task-related molecules are available (low-shot regime), where existing generative models fail to learn the target distribution.
Approach
Adapts textual inversion from images to a frozen text-to-molecule model using multi-level tokens (one shared token, K unsupervised intermediate cluster tokens, and per-molecule detail tokens), then generates novel molecules by interpolating intermediate and detail embeddings.
Key results
- On QM9, HI-Mol with 2% of training data reaches FCD 0.430 versus 0.585 for STGG trained on the full split.
- On HIV, HI-Mol improves Active ratio to 11.4% from 3.7% (PS-VAE best prior) and FCD to 19.0 (16.6 with SMILES modification) versus 20.2 for STGG.
- In low-shot property prediction, HI-Mol achieves the highest average Delta ROC-AUC on all three MoleculeNet datasets at 16 and 32 shots.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use hierarchical textual inversion instead of a single shared token (ablation) | HI-Mol · HIV | FCD: 64.2 -> 19.0 | Helped |
| Add detail tokens to the shared token (ablation) | HI-Mol · HIV | Active ratio: 0.0 -> 10.2 | Helped |
| Add intermediate tokens to the shared and detail tokens (ablation) | HI-Mol · HIV | FCD: 20.3 -> 19.0 | Helped |
| Apply the invalid-SMILES modification algorithm (ablation) | HI-Mol · HIV | FCD: 19.0 -> 16.6 | Helped |
| Use full hierarchical tokens instead of detail tokens only (ablation) | HI-Mol · HIV | Active ratio: 5.4 -> 11.4 | Helped |
| Use 2% QM9 training data with HI-Mol instead of STGG trained on the full split | Molecular generation · QM9 | FCD: 0.585 -> 0.430 | Helped |
| Increase QM9 training data from 2% to 10% | HI-Mol · QM9 | FCD: 0.430 -> 0.398 | Helped |
| Use grammar correction on generated SMILES (ablation) | HI-Mol · BBBP | FCD: 11.2 -> 10.7 | Helped |
| Increase the number of clusters from K=10 to K=20 (ablation) | HI-Mol · QM9 (2% training data) | FCD: 0.434 -> 0.430 | Helped |
| Increase the number of clusters from K=10 to K=2,113 (ablation) | HI-Mol · QM9 (2% training data) | FCD: 0.434 -> 0.443 | Hurt |
| Use the training-style prompt rather than the selected interpolation prompt (ablation) | HI-Mol · QM9 (2% training data) | FCD: 0.434 -> 0.210 | Helped |
| Use the selected interpolation prompt rather than the training-style prompt (ablation) | HI-Mol · QM9 (2% training data) | Novelty: 47.5 -> 73.5 | Helped |
Surprising findings
- Naive textual inversion with a single shared token yields only 0.4% validity on HIV, and DreamBooth yields 0.0%.
- Using a single molecule-level detail token (K=2,113 intermediate tokens) degrades performance relative to K=10-30.
Limitations
- Relies on a large pre-trained molecular language model; improvements depend on the quality of that model.
- Validity on MoleculeNet is only 60.6% without the SMILES modification algorithm, which is a heuristic post-processing step.
- Hyperparameters such as K were not extensively tuned.
Experimental setup
Datasets: MoleculeNet HIV MoleculeNet BBBP MoleculeNet BACE QM9
Benchmarks: MoleculeNet HIV MoleculeNet BBBP MoleculeNet BACE QM9
Metrics: Active ratio FCD NSPDK Validity Uniqueness Novelty Delta ROC-AUC
Baselines: GDSS DiGress JT-VAE PS-VAE MiCaM CRNN STGG DEG GraphAF GraphDF MoFlow EDP-GNN
Builds on: textual inversion MolT5 DreamBooth
Closest papers by meaning
- Text-Guided Molecule Generation with Diffusion Language Model (AAAI 2024 · similarity 0.87)
- Atomas: Hierarchical Adaptive Alignment on Molecule-Text for Unified Molecule Understanding and Generation (ICLR 2025 · similarity 0.87)
- Domain-Agnostic Molecular Generation with Chemical Feedback (ICLR 2024 · similarity 0.86)
- MolVision: Molecular Property Prediction with Vision Language Models (NeurIPS 2025 · similarity 0.86)
- FlashMol: High-Quality Molecule Generation in as Few as Four Steps (NeurIPS 2026 · similarity 0.86)
- Hierarchical Graph Tokenization for Molecule-Language Alignment (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.