Research map / Image generation / Autoregressive and latent generation
NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NExT-Mol, which pairs a 960M SELFIES language model (MoLlama) pretrained on 1.8B molecules with a relational-attention diffusion transformer (DMT) for 3D conformer prediction, and transfers MoLlama's 1D representations into DMT.
Problem
3D molecule generation via diffusion models often yields invalid molecules and is limited by scarce 3D training data, while 1D SELFIES language models guarantee validity and can exploit billion-scale 1D datasets.
Approach
Generate a 1D SELFIES molecule with a large autoregressive LM, then predict its 3D conformer with a diffusion transformer (DMT) using relational multi-head self-attention over atom and pair features; transfer LM representations to DMT through a cross-modal projector and staged training.
Key results
- NExT-Mol reduces 3D FCD on GEOM-DRUGS de novo generation from 14.69 (DMT-B) to 14.32 with MoLlama; 2D FCD improves from 0.655 to 0.334 over MolGen (49% relative), and the 3D FCD gain is reported as 26% relative over the best baseline.
- On QM9-2014 conditional generation, NExT-Mol achieves an average relative MAE improvement of about 13% over JODO across six quantum properties.
- DMT-B achieves 85.4 COV-R mean on GEOM-DRUGS (vs 84.0 for MCF-B), and DMT-L (150M) reaches 85.8 COV-R, with MoLlama integration raising DMT-L to 87.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MoLlama + DMT-B for de novo 3D generation instead of MolGen | GEOM-DRUGS de novo 3D generation · GEOM-DRUGS | 2D FCD: 0.655 -> 0.334 | Helped |
| use NExT-Mol rather than JODO | QM9-2014 conditional 3D generation · QM9-2014 | MAE Δε: 335 -> 297 | Helped |
| add MoLlama's pretrained 1D representations (ablation) | DMT-B · GEOM-DRUGS conformer prediction | COV-R mean (%): 85.4 -> 86.1 | Helped |
| add MoLlama's pretrained 1D representations (ablation) | DMT-B · GEOM-QM9 conformer prediction | AMR-R mean: 0.090 -> 0.083 | Helped |
| remove randomized SELFIES augmentation (ablation) | MoLlama · QM9-2014 1D molecule generation | V&U&N: 0.802 -> 0.395 | Hurt |
| remove randomized SELFIES augmentation (ablation) | MoLlama · QM9-2014 1D molecule generation | FCD: 0.070 -> 0.074 | Hurt |
| remove MoLlama pretraining (ablation) | MoLlama · GEOM-DRUGS 1D molecule generation | FCD: 0.334 -> 0.586 | Hurt |
| remove MoLlama pretraining (ablation) | MoLlama · GEOM-DRUGS 1D molecule generation | V&U&N: 0.945 -> 0.974 | Mixed |
| remove random rotation augmentation (ablation) | DMT-B · GEOM-QM9 conformer prediction | AMR-R mean: 0.090 -> 0.095 | Hurt |
| use a linear rather than cosine noise schedule at inference (ablation) | DMT-B · GEOM-DRUGS conformer prediction | COV-R mean (%): 85.4 -> 62.7 | Hurt |
| use a polynomial rather than cosine noise schedule at inference (ablation) | DMT-B · GEOM-DRUGS conformer prediction | COV-R mean (%): 85.4 -> 84.9 | No clear effect |
| increase batch size from 256 to 512 (ablation) | DMT-B · GEOM-DRUGS conformer prediction | COV-R mean (%): 85.4 -> 85.1 | Hurt |
Surprising findings
- Canonical SELFIES works better than randomized SELFIES when integrating MoLlama representations into DMT, contrary to the augmentation benefit seen for 1D generation.
- Pretraining MoLlama without pretraining gives higher novelty (V&U&N) but worse FCD, SNN, and Scaf, indicating a trade-off.
Limitations
- Adding MoLlama increases training cost substantially (210 vs 52 seconds per epoch for DMT-B on A100).
- Performance drops notably for molecules with unseen scaffolds, and transfer only partly mitigates this.
- DMT's pair representation has O(N^2) memory cost, which may limit very large molecules.
Experimental setup
Datasets: ZINC-15 GEOM-DRUGS GEOM-QM9 QM9-2014 MoleculeNet
Benchmarks: GEOM-DRUGS GEOM-QM9 QM9-2014 MoleculeNet FreeSolv MoleculeNet ESOL MoleculeNet Lipo MoleculeNet QM7
Metrics: FCD atom stability molecule stability validity & uniqueness & novelty SNN Frag Scaf bond length MMD bond angle MMD dihedral angle MMD
Baselines: CDGS JODO MiDi EQGAT-diff MolGPT MolGen EDM G-SchNet G-SphereNet MDM GeoLDM EEGSDE
Builds on: Llama-2 Torsional Diffusion MCF AlphaFold3 EDM
Closest papers by meaning
- MolHIT: Advancing Molecular-Graph Generation with Hierarchical Discrete Diffusion Models (NeurIPS 2026 · similarity 0.89)
- Geometric-Facilitated Denoising Diffusion Model for 3D Molecule Generation (AAAI 2024 · similarity 0.88)
- Navigating the Design Space of Equivariant Diffusion-Based Generative Models for De Novo 3D Molecule Generation (ICLR 2024 · similarity 0.88)
- BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning (AAAI 2025 · similarity 0.88)
- FlashMol: High-Quality Molecule Generation in as Few as Four Steps (NeurIPS 2026 · similarity 0.88)
- Towards 3D Molecule-Text Interpretation in Language Models (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.