Research map / Probabilistic and scientific modeling / Molecules and proteins
Energy Loss Functions for Physical Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes energy loss functions derived from reverse KL between a Boltzmann distribution centered on data and the model prediction, applied to atomistic regression/diffusion models and spin-glass ground-state prediction.
Problem
Standard losses such as MSE and cross-entropy are not grounded in the physics of the system; they penalize physically correct but symmetry-related predictions and yield poor gradients for physical configuration prediction and generation.
Approach
Obtain losses as energy differences by minimizing reverse KL between the model and a Boltzmann distribution around each data sample, using approximate physical energies (quadratic pair-distance potentials for atoms, local-field energy for spins) that are invariant to rigid motions and permutations.
Key results
- On QM9 with GDM-aug, energy loss raises molecule stability from 83.7% (MSE) to 89.8% and validity from 93.6% to 97.7%.
- On GEOM-Drugs with GDM-aug, molecule stability rises from 0.8% (MSE) to 24.6% (energy loss).
- On spin ground-state prediction, local energy loss gives test energy 45.6 versus 58.8 for cross-entropy and 49.87 for margin loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace MSE with exponential-coefficient energy loss | GDM-aug · QM9 | Molecule stability (%): 83.7 ± 2.3 -> 89.8 ± 2.8 | Helped |
| Replace MSE with exponential-coefficient energy loss | GDM-aug · QM9 | Validity (%): 93.6 ± 1.7 -> 97.7 ± 1.4 | Helped |
| Replace MSE with energy loss | GDM-aug · GEOM-Drugs | Molecule stability (%): 0.8 -> 24.6 | Helped |
| Replace cross-entropy with local-energy loss | CNN spin ground-state predictor · Spin ground-state prediction, test | Test energy: 58.8 ± 0.8 -> 45.6 ± 1.6 | Helped |
| Replace margin loss with local-energy loss | CNN spin ground-state predictor · Spin ground-state prediction, test | Test energy: 49.87 ± 1.5 -> 45.6 ± 1.6 | Helped |
| Add exponential distance-dependent spring coefficients rather than inverse-square coefficients (ablation) | GDM-aug energy loss · QM9 | Molecule stability (%): 84.6 ± 1.8 -> 89.8 ± 2.8 | Helped |
| Add exponential distance-dependent spring coefficients rather than inverse-distance coefficients (ablation) | GDM-aug energy loss · QM9 | Molecule stability (%): 84.5 ± 2.1 -> 89.8 ± 2.8 | Helped |
| Add exponential distance-dependent spring coefficients rather than constant coefficients (ablation) | GDM-aug energy loss · QM9 | Molecule stability (%): 83.6 ± 1.5 -> 89.8 ± 2.8 | Helped |
| Use exponential rather than inverse-distance energy coefficients (ablation) | JODO · QM9 | FCD: 0.892 -> 1.495 | Hurt |
| Use exponential rather than inverse-distance energy coefficients (ablation) | JODO · QM9 | Molecule stability (%): 94.3 -> 96.6 | Helped |
| Replace full energy graph with sparse rigid graph (ablation) | Energy-loss molecular diffusion model · QM9 | Molecule stability (%): 89.8 ± 2.8 -> 89.1 ± 0.9 | No clear effect |
Surprising findings
- Energy loss with a non-equivariant architecture gives more improvement than using an equivariant architecture, at negligible computational cost.
- Energy-based variants with physically motivated coefficients performed better empirically even though the variance/bias theory only covers constant coefficients.
Limitations
- Correct score is recovered only at low noise levels; higher noise requires explicit correction.
- Choice of energy coefficients and surrogate energies remains ad hoc and requires sweeps per task.
- Symmetrized sparse graphs may become quadratic in size for large symmetry orbits.
Experimental setup
Datasets: QM9 GEOM-Drugs spin-glass Hamiltonians (16x16 grid, synthetic)
Benchmarks: QM9 GEOM-Drugs regular polygon shape prediction spin-glass ground states 16x16
Metrics: molecule stability atom stability validity uniqueness FCD MMD bond/angle/dihedral configuration energy shape quality score
Baselines: MSE Kabsch align cross-entropy margin loss EDM JODO
Builds on: diffusion models Kabsch algorithm EDM JODO
Closest papers by meaning
- Cascade of phase transitions in the training of energy-based models (NeurIPS 2024 · similarity 0.86)
- Predicting the Energy Landscape of Stochastic Dynamical System via Physics-informed Self-supervised Learning (ICLR 2025 · similarity 0.86)
- Physics-Informed Weakly Supervised Learning For Interatomic Potentials (ICML 2025 · similarity 0.85)
- BoltzNCE: Learning likelihoods for Boltzmann Generation with Stochastic Interpolants and Noise Contrastive Estimation (NeurIPS 2025 · similarity 0.85)
- The dark side of the forces: assessing non-conservative force models for atomistic machine learning (ICML 2025 · similarity 0.84)
- Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular Systems (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.