Research map / Large language models / Language, safety and interpretability
Invariant Tokenization of Crystalline Materials for Language Model Enabled Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Mat2Seq, a tokenization that maps 3D crystals to unique, SE(3)- and periodic-invariant, complete 1D sequences (Niggli-reduced cell, space-group irreducible atoms) so a GPT-style LM can generate crystals without CIF augmentation.
Problem
Converting 3D crystal structures into 1D sequences for language models is not unique or invariant under SE(3) and periodic transformations when CIF files are used, so the same crystal can yield many different sequences.
Approach
Build a canonical unit cell (Niggli reduction, origin selected by atomic number and local densities) and encode it as lattice parameters, space group symbol, and fractional coordinates of irreducible atoms, yielding a unique and complete sequence per crystal that needs no data augmentation.
Key results
- On MPTS-52 one-shot generation, Mat2Seq achieves 23.1% match rate vs 12.2% for DiffCSP (about 89% relative improvement).
- With 20 shots on MP-20, Mat2Seq-small reaches RMSE 0.039 vs DiffCSP 0.049 while generating about 3x faster (2.1 s vs 7.3 s per crystal).
- Conditioned on band gap 0-0.5 eV, 83.6% of generated crystals have predicted band gap below 0.5 eV; conditioned on 3.5-4.0 eV, 90.7% exceed 3.0 eV.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Mat2Seq’s invariant crystal sequence representation instead of CIF sequences | Crystal language models · MP20 uniqueness verification | success rate: 0% -> 100% | Helped |
| Use Mat2Seq instead of CrystaLLM’s CIF representation | Crystal structure generation · Perov-5, Carbon-24, MP20, and MPTS-52; one shot and 20 shots | match rate | Helped |
| Use Mat2Seq instead of DiffCSP | Crystal structure generation · MPTS-52 one shot | match rate: +89% improvement | Helped |
| Use Mat2Seq instead of DiffCSP | Crystal structure generation · MPTS-52 20 shots | RMSE: 50% decrease | Helped |
| Reduce model size from 200 M to 25 M parameters (ablation) | Mat2Seq · MP20 test set | RMSE: 0.037 -> 0.039 | No clear effect |
| Reduce model size from 200 M to 25 M parameters (ablation) | Mat2Seq · MP20 test set | generation speed: 5.7 s -> 2.1 s | Helped |
| Increase sampling temperature from 1.35 to 1.65 (ablation) | Mat2Seq · MP20 random generation | structural validity: 94.2 % -> 88.6 % | Hurt |
| Increase sampling temperature from 1.35 to 1.65 (ablation) | Mat2Seq · MP20 random generation | S.U.N. rate: 2.0% -> 3.2% | Helped |
| Increase sampling temperature from 1.35 to 1.65 (ablation) | Mat2Seq · MP20 random generation | stability rate, E_hull < 0.0: 4.10% -> 4.50% | Helped |
| Condition generation toward band gap 0 to 0.5 eV | Mat2Seq · Generated crystals | success rate for band gap < 0.5 eV: 83.6% | Helped |
| Condition generation toward band gap 3.5 to 4.0 eV | Mat2Seq · Generated crystals | success rate for band gap > 3.0 eV: 90.7% | Helped |
| Use Mat2Seq instead of CrystaLLM | Crystal generation for recently discovered literature compositions · Recent literature challenge set | validity: 23.7% ± 2.5% -> 27.3% ± 1.4% | Helped |
Surprising findings
- Standard CIF-based representation achieves 0% uniqueness success on MP20 under shifted periodic boundaries, while CIF with symmetry reaches 30%.
- Mat2Seq's generated Eu2FeGe2OS6 structure is a reflected version of the ground truth, illustrating reflection sensitivity of the representation.
Limitations
- Cannot be directly applied to other atomic systems such as molecules and proteins.
- Modeling disordered materials remains an open challenge.
- Larger-scale training with more stable crystals could improve robustness, which requires more compute.
Experimental setup
Datasets: Perov-5 Carbon-24 MP-20 MPTS-52 Materials Project OQMD NOMAD JARVIS-DFT
Benchmarks: Perov-5 Carbon-24 MP-20 MPTS-52 MP-20 random generation (FlowMM protocol) 10 recently discovered literature crystals JARVIS-DFT band gap conditioning
Metrics: match rate RMSE validity stability (E_hull via DFT) S.U.N. rate hit rate success rate for band gap uniqueness novelty
Baselines: CrystaLLM CDVAE DiffCSP FlowMM MatterGen
Builds on: CrystaLLM ComFormer GPT-2 Niggli cell reduction
Closest papers by meaning
- Geometry Informed Tokenization of Molecules for Language Model Generation (ICML 2025 · similarity 0.89)
- Fine-Tuned Language Models Generate Stable Inorganic Materials as Text (ICLR 2024 · similarity 0.89)
- Generative Hierarchical Materials Search (NeurIPS 2024 · similarity 0.89)
- LLM Meets Diffusion: A Hybrid Framework for Crystal Material Generation (NeurIPS 2025 · similarity 0.89)
- Scalable Diffusion for Materials Generation (ICLR 2024 · similarity 0.88)
- SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.