Research map / Architectures and efficiency / Pruning and efficient tuning
ELoRA: Low-Rank Adaptation for Equivariant GNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ELoRA, a path-dependent low-rank adaptation for SO(3)-equivariant GNN interatomic potentials (MACE) that preserves equivariance during parameter-efficient fine-tuning and improves data efficiency.
Problem
Standard PEFT methods such as LoRA and adapters break SO(3) equivariance when applied to equivariant pre-trained interatomic potentials, and full-parameter fine-tuning is costly and can cause catastrophic forgetting.
Approach
Decompose weight updates along each tensor-product path (l1, l2, l3) into a low-rank product B A, so that the low-rank update respects the path constraints of the Clebsch-Gordan tensor product and remains SO(3)-equivariant; the update is mergeable into the original weights at inference.
Key results
- On rMD17 with 50 training configurations, ELoRA improves energy accuracy by 25.5% and force accuracy by 23.7% over full-parameter fine-tuning of MACE-OFF.
- Across 10 inorganic datasets, ELoRA on MACE-MP improves average energy RMSE by 12.3% and force RMSE by 14.4% over full-parameter fine-tuning.
- In data-efficiency tests on rMD17-aspirin, ELoRA reduces the required data by 42% (energy) and 44% (force) to reach similar error as full fine-tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ELoRA instead of full-parameter fine-tuning | MACE-MP · 10 inorganic datasets | energy prediction accuracy: 12.3% | Helped |
| Use ELoRA instead of full-parameter fine-tuning | MACE-MP · 10 inorganic datasets | force prediction accuracy: 14.4% | Helped |
| Use ELoRA instead of full-parameter fine-tuning | MACE-OFF · rMD17 | energy prediction accuracy: 25.5% | Helped |
| Use ELoRA instead of full-parameter fine-tuning | MACE-OFF · rMD17 | force prediction accuracy: 23.7% | Helped |
| Use ELoRA instead of full-parameter fine-tuning | MACE-OFF · 3BPA at 300 K, 600 K, and 1200 K | energy RMSE: 0.3 (meV) to 2.7 (meV) | Helped |
| Use ELoRA instead of full-parameter fine-tuning | MACE-OFF · 3BPA at 300 K, 600 K, and 1200 K | force RMSE: 0.3 (meV/Å) to 6.7 (meV/Å) | Helped |
| Use ELoRA instead of full-parameter fine-tuning | MACE-OFF · AcAc at 300 K and 600 K | energy prediction accuracy: 26.4% | Helped |
| Use ELoRA instead of full-parameter fine-tuning | MACE-OFF · AcAc at 300 K and 600 K | force prediction accuracy: 14.4% | Helped |
| Fine-tune the pre-trained MACE-OFF model | MACE-OFF · 3BPA | energy RMSE | Helped |
| Fine-tune the pre-trained MACE-OFF model | MACE-OFF · AcAc | energy RMSE | Helped |
| Increase ELoRA rank (ablation) | ELoRA · rMD17-aspirin and Cu | validation loss | Mixed |
| Use ELoRA rather than full-parameter fine-tuning across training-data sizes 50–1000 | MACE-OFF on rMD17-aspirin · rMD17-aspirin | energy and force MAE: 42% and 44% | Helped |
Surprising findings
- Standard PEFT methods such as LoRA are argued to inherently break equivariance, so they cannot be applied directly to equivariant potentials.
Limitations
- Theoretical analysis of generalization bounds is not rigorous and left as future work.
- Energy predictions on inorganic datasets remain slightly behind state-of-the-art models, attributed to architecture differences.
- Evaluated only with the MACE backbone and MACE pre-trained models.
Experimental setup
Datasets: rMD17 3BPA AcAc SPICE MPTrj SSE-PBE H2O-PD Ag-Au-PBE Al-Mg-Cu Cu Sn Ti
Benchmarks: rMD17 3BPA AcAc 10 inorganic DP-GEN datasets
Metrics: MAE energy (meV) MAE force (meV/Å) RMSE energy (meV/atom) RMSE force (meV/Å)
Baselines: full-parameter fine-tuning ACE NequIP BOTNet Allegro DPA2 PACE MACE from scratch MACE-MP zero-shot MACE-OFF zero-shot
Builds on: LoRA MACE AdapterGNN
Closest papers by meaning
- Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing (NeurIPS 2024 · similarity 0.89)
- The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains (NeurIPS 2024 · similarity 0.88)
- Learning Equivariant Non-Local Electron Density Functionals (ICLR 2025 · similarity 0.88)
- Efficient and Scalable Density Functional Theory Hamiltonian Prediction through Adaptive Sparsity (ICML 2025 · similarity 0.87)
- Long-Short-Range Message-Passing: A Physics-Informed Framework to Capture Non-Local Interaction for Scalable Molecular Dynamics Simulation (ICLR 2024 · similarity 0.87)
- Compact SO(3) Equivariant Atomistic Foundation Models via Structural Pruning (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.