Research map / Probabilistic and scientific modeling / Molecules and proteins
MLIP Arena: Advancing Fairness and Transparency in Machine Learning Interatomic Potentials via an Open, Accessible Benchmark Platform
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MLIP Arena, an open benchmark platform that evaluates foundation machine learning interatomic potentials on physical soundness, MD stability/reactivity, distribution-shift robustness, and thermodynamic properties, reducing reliance on DFT reference errors.
Problem
Existing MLIP benchmarks rely on error-based regression metrics tied to specific DFT references, which suffer from data leakage, poor transferability, and fail to reflect physical soundness and downstream utility.
Approach
Replace DFT-reference regression metrics with reference-agnostic physics and utility tests (PEC smoothness, short-range repulsion, conservation deviation on diatomics; EOS and energy-volume scans; NVT/NPT MD stability; hydrogen combustion reactivity; energy conservation and rotational equivariance under differential-entropy shifts; NEB vacancy symmetry; MOF CO2 adsorption; 2D stability; BaZrO3 pha
Key results
- On the WBM EOS benchmark, MACE-MPA ranked best on derivative flips (1.037) and tortuosity (1.005), while ORBv2 was worst with 7 missing predictions.
- ORBv2 was the fastest MD model and had the best scaling exponent, but could not sustain pressures up to 500 GPa without an explicit short-range repulsion.
- Direct force prediction models (ORB, ORBv2, EquiformerV2) showed energy conservation deviations and rotational force errors correlating with differential entropy, while gradient-based models did not; highest macro F1 for 2D stability was 0.420 (MACE family).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate force-field performance with physical and downstream-task benchmarks rather than only static DFT regression metrics | Foundation MLIPs in MLIP Arena · Real-world atomistic modeling tasks | Physical consistency and predictive utility | Mixed |
| Use gradient-based rather than direct force prediction | MLIP force prediction models · NVE MD under distribution shift | Energy conservation versus differential entropy | Helped |
| Use explicitly rotationally equivariant architectures rather than non-equivariant ORB/ORBv2 | MLIP force prediction models · MPTrj structures across differential entropy | Rotational force MAE: 0.0 | Helped |
| Use ORBv2 | MLIP models · RM24 NVT and NPT stability simulations | MD speed and scaling | Helped |
| Rely on ORBv2 without an explicit short-range core repulsive potential | ORBv2 · RM24 high-pressure simulations | Valid MD runs | Hurt |
| Use MatterSim rather than M3GNet | MLIP models · RM24 extreme-condition MD | Stability and runtime performance | Helped |
| Use MACE-MP(M) or EquiformerV2(OC20) for hydrogen combustion | MLIP models · Hydrogen combustion annealing MD | Ignition in experimental flame-temperature region: 2380 K; 3000 K | Helped |
| Use ORB or ORBv2 rather than reactive models for hydrogen combustion | ORB and ORBv2 · Hydrogen combustion annealing MD | Water production and reaction enthalpy | Hurt |
| Predict forces directly rather than derive them from an energy gradient | Direct-force MLIP models · Hydrogen combustion MD | Center-of-mass drift: >10^{2} Å; ∼ 2.4 Å; 10^{-4} Å | Hurt |
| Use MACE-MP(M), MatterSim, or ORBv2 rather than M3GNet, CHGNet, or SevenNet | MLIP models · Vacancy migration CI-NEB | NEB relaxation robustness | Helped |
| Use MACE-family models for dynamical stability screening | MLIP models · 2D materials | Macro F1 score: 0.420 and 0.412 | Mixed |
| Use a non-equivariant architecture for vacancy migration | MLIP models · FCC and HCP elemental crystals | Migration profile symmetry | No clear effect |
Surprising findings
- Many top-ranked models on bulk EOS and Matbench Discovery performed poorly on homonuclear diatomic pairwise interactions, suggesting many-body error cancellation in bulk benchmarks.
- Equivariant models were not uniformly slower in MD; CHGNet and M3GNet slowed down because molecules condensed into droplets, increasing neighbor edges.
Limitations
- Moving away from DFT references makes direct comparison with in-distribution test errors harder.
- Conservation deviation definition is only validated for diatomic interactions; many-body extension left open.
- Some models (e.g., CHGNet, M3GNet, EquiformerV2) could not finish the hydrogen combustion trajectories, and validity criteria are lenient.
Experimental setup
Datasets: WBM RM24 (random mixture dataset, constructed from Materials Project v2024.12.18) MPTrj vacancy diffusion database C2DB
Benchmarks: MLIP Arena EOS benchmark (WBM 1000 structures) energy-volume scan homonuclear diatomic PECs MD stability on RM24 (NVT/NPT) hydrogen combustion annealing energy conservation and rotational equivariance under distribution shift vacancy migration NEB CO2 adsorption in MOFs 2D material dynamical stability BaZrO3 phase transition
Metrics: derivative flips tortuosity Spearman's coefficient energy jump conservation deviation missing predictions MD steps per second force MAE under rotation barrier asymmetry macro F1
Baselines: MACE-MPA MACE-MP(M) eSEN MatterSim CHGNet SevenNet M3GNet ORBv2 ORB eSCN(OC20) eqV2(OMat) ALIGNN
Builds on: Matbench Discovery Birch-Murnaghan equation of state QUESTS descriptors Prefect Atomic Simulation Environment
Closest papers by meaning
- Learning Smooth and Expressive Interatomic Potentials for Physical Property Prediction (ICML 2025 · similarity 0.90)
- Physics-Informed Weakly Supervised Learning For Interatomic Potentials (ICML 2025 · similarity 0.89)
- FlashTP: Fused, Sparsity-Aware Tensor Product for Machine Learning Interatomic Potentials (ICML 2025 · similarity 0.88)
- Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials (NeurIPS 2026 · similarity 0.88)
- Colour me shocked: Exact Molecular Hessians from MLIPs in O(N) time using sparse differentiation! (NeurIPS 2026 · similarity 0.87)
- EDBench: Large-Scale Electron Density Data for Molecular Modeling (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.