Research map / Architectures and efficiency / RNNs and normalization
The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Systematically studies scaling in neural network interatomic potentials and proposes EScAIP, a scalable attention-based NNIP operating on rotation-invariant scalar features with neighbor-level multi-head self-attention, achieving SOTA on OC20, OC22, MPTrj, and SPICE with large speedups.
Problem
Neural network interatomic potentials rely on heavy physical symmetry constraints (e.g., rotational equivariance via tensor products) that limit computational efficiency and scalability with model size and data.
Approach
Replace equivariant tensor-product constructions with invariant scalar features (including a rotation-invariant bond-orientational order embedding) and scale parameters via multi-head self-attention applied on neighbor-level message tensors, implemented with optimized attention kernels; the model learns rotational behavior from data.
Key results
- EScAIP-Large (317M) reaches OC20 All+MD test force MAE 12.65 meV/Å and energy MAE 215 meV, outperforming EquiformerV2 (153M, 14.20 meV/Å).
- EScAIP achieves about 10x faster inference and 5x lower memory than EquiformerV2 of comparable size on OC20 (V100 32G; Small 107.04 vs EquiformerV2 34.55 samples/sec at 31M).
- On Matbench-Discovery, EScAIP reaches F1 0.782 and MAE 38 meV/atom, the best among compliant models listed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale EScAIP from Small to Large | EScAIP · OC20 All+MD validation | Energy MAE: 229 -> 211 | Helped |
| Scale EScAIP from Small to Large | EScAIP · OC20 All+MD validation | Force MAE: 15.07 -> 12.17 | Helped |
| Use EScAIP-Medium instead of EquiformerV2 | OC20 2M validation · OC20 2M validation | Energy MAE: 285 -> 254 | Helped |
| Use EScAIP-Medium instead of EquiformerV2 | OC22 test · OC22 test | Energy MAE: 462 -> 473 | Hurt |
| Use EScAIP-Medium instead of EquiformerV2 | OC22 test · OC22 test | Force MAE: 27.1 -> 25.73 | Helped |
| Replace EquiformerV2 with EScAIP-Medium | OC20 inference · OC20 | Inference speed: 7.92 -> 74.77 Sample/Sec | Helped |
| Replace EquiformerV2 with EScAIP-Medium | OC20 inference · OC20 | Inference memory: 5.24 -> 1.28 GB/Sample | Helped |
| Use EScAIP instead of EquiformerV2 | Matbench-Discovery (MPTrj) · Matbench-Discovery | MAE: 42 -> 38 | Helped |
| Use EScAIP instead of MACE | SPICE · SPICE PubChem | Force MAE: 14.75 -> 5.86 | Helped |
| Increase attention parameters (More AT) rather than scaling spherical channels (ablation) | Parameter-controlled EquiformerV2 · OC20 S2EF 2M | Energy and force MAE | Helped |
| Increase parameters in attention mechanisms (AT) rather than spherical channels (SC) (ablation) | Parameter-controlled EquiformerV2 · OC20 S2EF 2M | Energy and force MAE | Helped |
| Increase parameters through attention mechanisms rather than increasing rotation order (ablation) | EquiformerV2 · OC20 S2EF 2M with increasing training data | Force MAE scaling slope | Helped |
Surprising findings
- Increasing attention parameters was more effective than increasing rotation order L, and an invariant L=0 model with BOO features matched the L=6 model.
- Although the model is not rotationally equivariant at initialization (cosine similarity ~0.2), after training it reaches cosine similarity of 0.998 or higher, suggesting symmetry is learned from data.
Limitations
- Limited error bars: error bars not reported for every experiment due to computational cost.
- Some baselines are reported from prior papers, and several comparisons use different parameter counts or training settings.
- The model is not rotationally equivariant by construction; equivariance is only learned approximately from data.
Experimental setup
Datasets: OC20 OC22 MPTrj SPICE MD22
Benchmarks: OC20 OC22 Matbench-Discovery SPICE MD22
Metrics: energy MAE force MAE F1 DAF RMSE R2 inference speed (samples/sec) memory (GB/sample) h(r) MAE cosine similarity (equivariance)
Baselines: EquiformerV2 eSCN SCN GemNet-OC GemNet-dT MACE SevenNet ORB MPTrj MACE-OFF23 VisNet-LSRM sGDML
Builds on: EquiformerV2 GemNet-OC Transformer self-attention Bond-orientational order
Closest papers by meaning
- FlashTP: Fused, Sparsity-Aware Tensor Product for Machine Learning Interatomic Potentials (ICML 2025 · similarity 0.89)
- Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials (NeurIPS 2026 · similarity 0.89)
- Colour me shocked: Exact Molecular Hessians from MLIPs in O(N) time using sparse differentiation! (NeurIPS 2026 · similarity 0.88)
- ELoRA: Low-Rank Adaptation for Equivariant GNNs (ICML 2025 · similarity 0.88)
- Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing (NeurIPS 2024 · similarity 0.88)
- Physics-Informed Weakly Supervised Learning For Interatomic Potentials (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.