Research map / Architectures and efficiency / CNN design
Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LieLAC, an energy-based (weighted closed) canonicalization that uses only Lie algebra generators to make pre-trained models invariant/equivariant, demonstrated on MNIST affine/homography invariance and PDE neural solvers (Heat, Burgers, Allen-Cahn).
Problem
Making pre-trained, non-equivariant models (including physics-informed neural operators) equivariant under non-compact Lie point symmetry groups whose full group structure is often unavailable and which are incompatible with existing equivariant architectures.
Approach
Energy-based canonicalization that minimizes an energy (e.g., VAE ELBO plus adversarial regularizer, or distance to training domain) over the closures of group orbits, optimized via Lie algebra coordinate/group descent, so only infinitesimal generators are needed; theoretically extends frames and canonicalizations to non-compact groups via weighted closed canonicalizations.
Key results
- On affNIST, LieLAC with a CNN reaches 0.972 accuracy vs 0.629 for the plain CNN and 0.943 for affConv.
- On homNIST, LieLAC with a CNN reaches 0.960 vs 0.644 for the plain CNN and 0.927 for homConv.
- On the Allen-Cahn equation, LieLAC fine-tuned Poseidon reduces average error to 10.79e-4 from 41.35e-4 for the pre-trained Poseidon, with OOD error 11.34e-4 vs 75.76e-4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply affine canonicalization to CNN inputs | CNN · affNIST | accuracy: 0.629 -> 0.972 | Helped |
| apply affine canonicalization to CNN inputs | CNN · MNIST | accuracy: 0.985 -> 0.979 | Hurt |
| apply homography canonicalization to CNN inputs | CNN · homNIST | accuracy: 0.644 -> 0.960 | Helped |
| canonicalize DeepONet inputs using heat-equation symmetries | DeepONet · Heat ID | L2 relative error: $0.0498 \pm 0.0072$ -> $\mathbf{0.0443 \pm 0.0027}$ | Helped |
| canonicalize DeepONet inputs using heat-equation symmetries | DeepONet · Heat OOD | L2 relative error: $0.6572 \pm 0.1235$ -> $\mathbf{0.0435 \pm 0.0017}$ | Helped |
| canonicalize DeepONet inputs using heat-equation symmetries | DeepONet · Heat OOD, data augmentation | L2 relative error: $0.0687 \pm 0.0044$ -> $\mathbf{0.0500 \pm 0.0003}$ | Helped |
| canonicalize DeepONet inputs using Burgers-equation symmetries | DeepONet · Burgers ID | L2 relative error: $\mathbf{0.0832 \pm 0.0547}$ -> ${0.0916 \pm 0.0632}$ | Hurt |
| canonicalize DeepONet inputs using Burgers-equation symmetries | DeepONet · Burgers OOD | L2 relative error: $0.8369 \pm 0.0987$ -> $\mathbf{0.1006 \pm 0.0637}$ | Helped |
| apply Allen–Cahn symmetry canonicalization | Poseidon · ACE OOD | test error (× E-04): \num{75.76} $\pm$ \num{6.80} -> \num{29.19} $\pm$ \num{5.64} | Helped |
| apply Allen–Cahn symmetry canonicalization | Poseidon · ACE ID | test error (× E-04): \num{6.93} $\pm$ \num{2.83} -> \num{16.69} $\pm$ \num{5.42} | Hurt |
| add fine-tuning on canonicalized data (ablation) | LieLAC [Poseidon] · ACE OOD | test error (× E-04): \num{29.19} $\pm$ \num{5.64} -> \textbf{\num{11.34} $\pm$ \num{2.92}} | Helped |
| add fine-tuning on canonicalized data (ablation) | LieLAC [Poseidon] · ACE ID | test error (× E-04): \num{16.69} $\pm$ \num{5.42} -> \num{10.23} $\pm$ \num{3.36} | Helped |
Surprising findings
- Canonicalization without fine-tuning already substantially reduces out-of-distribution error for the PDE solvers.
- Energy-based canonicalization slows inference by 5-30x.
Limitations
- Energy-based canonicalization has slow inference (5-30x slowdown).
- Implementing Lie point symmetries in existing neural operators is hard due to hard-coded boundary conditions.
- Energy design and minimizer alignment with training data rely on heuristics such as VAE ELBO and adversarial regularizers.
Experimental setup
Datasets: MNIST affNIST homNIST Heat equation simulated trajectories Burgers equation simulated trajectories Allen-Cahn equation trajectories
Benchmarks: MNIST affNIST homNIST Heat equation Burgers equation Allen-Cahn equation
Metrics: test accuracy L2 relative error test error (x1e-4)
Baselines: CNN affConv homConv DeepONet Poseidon data augmentation
Builds on: canonicalization frame averaging adversarial regularization DeepXDE
Closest papers by meaning
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.91)
- Explicit Discovery of Nonlinear Symmetries from Dynamic Data (ICML 2025 · similarity 0.88)
- Lie Neurons: Adjoint-Equivariant Neural Networks for Semisimple Lie Algebras (ICML 2024 · similarity 0.88)
- Learning Infinitesimal Generators of Continuous Symmetries from Data (NeurIPS 2024 · similarity 0.87)
- A Canonicalization Perspective on Invariant and Equivariant Learning (NeurIPS 2024 · similarity 0.87)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.