Research map / Label-efficient and robust learning / Recognition and long-tail
A Denoising Pre-training Framework for Accelerating Novel Material Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPF, an unsupervised denoising pre-training framework for crystal structures (masked atom types, perturbed atom positions and lattice constants), pre-trained on 380,743 generated crystals and fine-tuned for property prediction on JARVIS and Materials Project.
Problem
Crystal property prediction is limited by scarce, property-specific labeled data, leaving large amounts of unlabeled crystal structures unused.
Approach
Pre-train any periodic crystal GNN by reconstructing the native crystal from corrupted versions (masked atom types, Gaussian-noised atom positions and lattice constants), with masked atom-type prediction found to be the most effective perturbation and architecture-agnostic transfer to downstream tasks.
Key results
- DPF with 30% atom-type masking improves JARVIS Total Energy MAE from 0.035 (Matformer) to 0.0289 eV/atom and Ehull from 0.064 to 0.036 eV.
- DPF with 10% atom-position noise gains 4.2% on Bulk Moduli per the text; DPF outperforms Matformer on 8 of 9 sub-tasks with atom-position noise.
- Lattice-constant and atom-position noise pre-training often degrades performance vs. baseline, while atom-type masking gives more stable gains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add denoising pre-training with masked atom types (γ = 30%) | Matformer · JARVIS | MAE, BandGap(OPT): 0.137 -> 0.118 | Helped |
| add denoising pre-training with masked atom types (γ = 30%) | Matformer · JARVIS | MAE, Total Energy: 0.035 -> 0.0289 | Helped |
| add denoising pre-training with masked atom types (γ = 30%) | Matformer · JARVIS | MAE, Ehull: 0.064 -> 0.036 | Helped |
| add denoising pre-training with masked atom types (γ = 30%) | CrysDiff · JARVIS | MAE, Bandgap(MBJ): 0.287 -> 0.311 | Hurt |
| add denoising pre-training with masked atom types | Matformer · Materials Project | MAE, Formation Energy: 0.021 -> 0.0196 | Helped |
| add denoising pre-training with masked atom types | Matformer · Materials Project | MAE, Shear Moduli: 0.073 -> 0.0734 | Hurt |
| change atom-type mask ratio from γ = 30% to γ = 50% (ablation) | DPF · JARVIS | MAE, Bandgap(MBJ): 0.311 -> 0.315 | Hurt |
| change atom-type mask ratio from γ = 30% to γ = 70% (ablation) | DPF · JARVIS | MAE, Bandgap(MBJ): 0.311 -> 0.316 | Hurt |
| pre-train with lattice-constant noise at β = 10% instead of masked atom types (ablation) | DPF · JARVIS | MAE, Bandgap(MBJ): 0.311 -> 0.329 | Hurt |
| pre-train with lattice-constant noise at β = 30% instead of masked atom types (ablation) | DPF · JARVIS | MAE, Bandgap(MBJ): 0.311 -> 0.329 | Hurt |
| pre-train with lattice-constant noise at β = 50% instead of masked atom types (ablation) | DPF · JARVIS | MAE, Bandgap(MBJ): 0.311 -> 0.326 | Hurt |
| pre-train with atom-position noise at α = 10% instead of masked atom types (ablation) | DPF · JARVIS | MAE, Bandgap(MBJ): 0.311 -> 0.310 | No clear effect |
Surprising findings
- Perturbing lattice constants or atom positions often hurts downstream performance relative to the baseline, unlike masked atom-type pre-training.
- Pre-training with coordinate/lattice noise makes fine-tuning loss curves fluctuate more as noise scale grows, attributed to KNN graph edges changing under perturbation.
Limitations
- Perturbation of lattice constants and atom positions gives unstable, sometimes worse results.
- Does not explore crystal structure generation downstream tasks.
- Pre-training data is generated by a generative tool rather than experimentally verified structures.
Experimental setup
Datasets: 380,743 crystal structures filtered from Merchant et al. 2023 JARVIS Materials Project
Benchmarks: JARVIS Materials Project
Metrics: MAE
Baselines: CGCNN SchNet MEGNet GATGNN ALIGNN Matformer CrysDiff CrysGNN
Builds on: Matformer CrysDiff masked language modeling
Closest papers by meaning
- A Diffusion-Based Pre-training Framework for Crystal Property Prediction (AAAI 2024 · similarity 0.90)
- Scalable Diffusion for Materials Generation (ICLR 2024 · similarity 0.88)
- Open Materials Generation with Stochastic Interpolants (ICML 2025 · similarity 0.88)
- LLM Meets Diffusion: A Hybrid Framework for Crystal Material Generation (NeurIPS 2025 · similarity 0.87)
- Vector Field Oriented Diffusion Model for Crystal Material Generation (AAAI 2024 · similarity 0.87)
- All that structure matches does not glitter (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.