Research map / 3D vision / 3D generation
Geometry Informed Tokenization of Molecules for Language Model Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Geo2Seq, which serializes 3D molecules into SE(3)-invariant 1D token sequences (canonical labeling via Nauty plus spherical coordinates relative to an equivariant frame), then trains GPT and Mamba LMs for unconditional and property-conditioned 3D molecule generation.
Problem
Generating 3D molecular geometries with language models requires a discrete, SE(3)-invariant tokenization of 3D point clouds, which prior work has largely left unexplored.
Approach
Reorder atoms by Nauty canonical labeling and encode each atom as [atom type, distance, polar angle, azimuth] relative to a global frame built from the first three canonically ordered non-collinear atoms, yielding an SE(3)-invariant, reversible 1D sequence that any autoregressive LM can model.
Key results
- On QM9, Geo2Seq with Mamba reaches 98.9% atom stability, 93.2% molecule stability and 97.1% validity, beating EDM and GEOLDM on molecule stability and validity.
- On GEOM-DRUGS, Geo2Seq with Mamba reaches 96.1% validity versus 99.3% for GEOLDM, and comparable atom stability (82.5 vs 84.4).
- For controllable generation on QM9 (Mamba), MAE on polarizability is 0.46 vs 2.37 for GEOLDM, with large gains across all six properties.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Geo2Seq with GPT instead of GEOLDM | QM9 random generation · QM9 | Molecule stability (%): 84.6$\pm$0.3 -> 90.3$\pm$0.1 | Helped |
| Use Geo2Seq with Mamba instead of GEOLDM | QM9 random generation · QM9 | Valid (%): 93.8$\pm$0.4 -> 97.1$\pm$0.2 | Helped |
| Use Geo2Seq with Mamba instead of GEOLDM | QM9 random generation · QM9 | Valid & unique (%): 92.7$\pm$0.5 -> 81.7$\pm$0.4 | Hurt |
| Use Geo2Seq with Mamba instead of GEOLDM | GEOM-DRUGS random generation · GEOM-DRUGS | Atom stability (%): 84.4 -> 82.5 | Hurt |
| Use Geo2Seq with Mamba instead of GEOLDM | GEOM-DRUGS random generation · GEOM-DRUGS | Valid (%): 99.3 -> 96.1 | Hurt |
| Use Geo2Seq with Mamba instead of GEOLDM | QM9 controllable generation · QM9 | MAE (Bohr³): 2.37 -> 0.46 | Helped |
| Use Geo2Seq with Mamba instead of GEOLDM | QM9 controllable generation · QM9 | MAE (meV; HOMO-LUMO gap Δε): 587 -> 98 | Helped |
| Use Geo2Seq with Mamba instead of GEOLDM | QM9 controllable generation · QM9 | MAE (meV; εHOMO): 340 -> 57 | Helped |
| Use Geo2Seq with Mamba instead of GEOLDM | QM9 controllable generation · QM9 | MAE (meV; εLUMO): 522 -> 71 | Helped |
| Use Geo2Seq with Mamba instead of GEOLDM | QM9 controllable generation · QM9 | MAE (D): 1.108 -> 0.164 | Helped |
| Use Geo2Seq with Mamba instead of GEOLDM | QM9 controllable generation · QM9 | MAE (cal/mol K; Cv): 1.025 -> 0.275 | Helped |
Surprising findings
- Geo2Seq's uniqueness on QM9 is lower (about 81%) than diffusion baselines, which the authors attribute to discretization limiting the search space.
Limitations
- Rounding coordinates to decimal places loses information and discretization hurts generalization over continuous values.
- Lower uniqueness on QM9 than point-cloud diffusion baselines.
- Only GPT and Mamba at modest scale; GEOM-DRUGS performance not beating the best diffusion baseline.
Experimental setup
Datasets: QM9 GEOM-DRUGS
Benchmarks: QM9 GEOM-DRUGS
Metrics: atom stability molecule stability validity valid and unique MAE of conditioned properties
Baselines: G-SchNet E-NFs GDM GDM-AUG EDM EDM-Bridge GEOLDM
Builds on: Nauty EDM GPT Mamba Top-k sampling
Closest papers by meaning
- Invariant Tokenization of Crystalline Materials for Language Model Enabled Generation (NeurIPS 2024 · similarity 0.89)
- 3D Molecule Generation from Rigid Motifs via $\mathrm{SE}(3)$ Flows (NeurIPS 2026 · similarity 0.89)
- Geometric Representation Condition Improves Equivariant Molecule Generation (ICML 2025 · similarity 0.88)
- 3D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modeling (ICLR 2025 · similarity 0.87)
- Hierarchical Graph Tokenization for Molecule-Language Alignment (ICML 2025 · similarity 0.87)
- Towards 3D Molecule-Text Interpretation in Language Models (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.