Research map / Image generation / Autoregressive and latent generation
A Diffusion-Based Pre-training Framework for Crystal Property Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CrysDiff, a diffusion-based pretrain-finetune framework that pretrains on unlabeled crystal structures via joint lattice/fractional-coordinate denoising and fine-tunes by diffusing on the target property, with EGNN backbone preserving O(3) and periodic translation symmetry.
Problem
Crystal property prediction from 3D structures is limited by scarce labeled data, since labels require expensive DFT simulations or lab experiments.
Approach
Uses a single EGNN-based joint diffusion model over lattice and fractional coordinates for self-supervised structure reconstruction pretraining, then reuses the same backbone by diffusing on the target property value (CARD-style) with the crystal structure fixed, with theoretical guarantees of O(3) invariance and periodic translation invariance.
Key results
- CrysDiff achieves the lowest MAE on all nine JARVIS-DFT properties, e.g., formation energy 0.029 vs 0.033 for Matformer and bulk modulus 9.875 vs 11.21 for Matformer.
- With sparse labeled data (20-80%), CrysDiff outperforms Matformer and CrysGNN across the tested splits on JARVIS-DFT.
- On OQMD-EXP with 80% experimental data for finetuning, CrysDiff reaches MAE 0.087 vs 0.096 for CrysGNN and 0.110 for CrysXPP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CrysDiff instead of ALIGNN | Crystal property prediction · JARVIS-DFT, Formation Energy | MAE: 0.033 -> 0.029 | Helped |
| Use CrysDiff instead of Matformer | Crystal property prediction · JARVIS-DFT, Bandgap (OPT) | MAE: 0.137 -> 0.131 | Helped |
| Use CrysDiff instead of Matformer | Crystal property prediction · JARVIS-DFT, Total Energy | MAE: 0.035 -> 0.034 | Helped |
| Use CrysDiff instead of Matformer | Crystal property prediction · JARVIS-DFT, Ehull | MAE: 0.064 -> 0.062 | Helped |
| Use CrysDiff instead of Matformer | Crystal property prediction · JARVIS-DFT, Bandgap (MBJ) | MAE: 0.302 -> 0.287 | Helped |
| Use CrysDiff instead of ALIGNN | Crystal property prediction · JARVIS-DFT, Bulk Modulus (Kv) | MAE: 10.40 -> 9.875 | Helped |
| Use CrysDiff instead of ALIGNN | Crystal property prediction · JARVIS-DFT, Shear Modulus (Gv) | MAE: 9.481 -> 9.193 | Helped |
| Use CrysDiff instead of CrysXPP | Crystal property prediction · JARVIS-DFT, SLME (%) | MAE: 5.110 -> 5.030 | Helped |
| Use CrysDiff instead of CrysXPP | Crystal property prediction · JARVIS-DFT, Spillage | MAE: 0.363 -> 0.358 | Helped |
| Use CrysDiff instead of CrysGNN | Formation energy prediction · OQMD-EXP, train on DFT and test on experimental data | MAE: 0.253 -> 0.211 | Helped |
| Jointly diffuse lattice and fractional coordinates during pre-training (ablation) | CrysDiff · JARVIS-DFT, Formation Energy and Bandgap (MBJ) | MAE | Helped |
| Use an EGNN denoiser (ablation) | CrysDiff · JARVIS-DFT, Formation Energy and Bandgap (MBJ) | MAE | Helped |
Surprising findings
- Several existing pretrain-finetune models (e.g., Pretrain-GNN, CrysXPP, CrysGNN) fail to beat the train-from-scratch Matformer on many properties.
- Freezing the pretrained encoder during fine-tuning (w/o UE) causes the largest ablation performance drop.
Limitations
- Pretraining and fine-tuning are computationally heavy with iterative diffusion sampling at inference; efficiency is not addressed.
- Evaluation limited to JARVIS-DFT and a small OQMD-EXP experimental set; no cross-dataset generalization shown.
- Reported improvements rely on single-split MAE without variance reporting.
Experimental setup
Datasets: Materials Project OQMD JARVIS-DFT OQMD-EXP
Benchmarks: JARVIS-DFT OQMD-EXP
Metrics: MAE
Baselines: CGCNN GATGNN ALIGNN Matformer Pretrain-GNN CrysXPP CrysGNN
Builds on: DDPM score-based generative modeling EGNN CARD GeoDiff
Closest papers by meaning
- A Denoising Pre-training Framework for Accelerating Novel Material Discovery (AAAI 2025 · similarity 0.90)
- A Periodic Bayesian Flow for Material Generation (ICLR 2025 · similarity 0.89)
- Vector Field Oriented Diffusion Model for Crystal Material Generation (AAAI 2024 · similarity 0.89)
- LLM Meets Diffusion: A Hybrid Framework for Crystal Material Generation (NeurIPS 2025 · similarity 0.88)
- Scalable Diffusion for Materials Generation (ICLR 2024 · similarity 0.88)
- Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.