Research map / Architectures and efficiency / CNN design
LieTransformer: Equivariant Self-Attention for Lie Groups
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LieTransformer (equivariant self-attention) instead of non-invariant SetTransformer on shape counting | shape counting point clouds · Shape counting | test accuracy: 0.75 vs 0.58 (T2 setting) | Helped |
| Add SO(3) rotation data augmentation to T(3)-invariant LieTransformer (ablation) | QM9 LieTransformer-T3 · QM9 | MAE: T3 alpha .179 -> .082 with aug | Helped |
| Make model SE(3)-invariant (rotation invariance) rather than T(3) without augmentation (ablation) | QM9 LieTransformer · QM9 | MAE: SE3 models outperform T3 without augmentation on 10/12 tasks | Helped |
| Use LieTransformer instead of LieConv (attention vs convolution) for SE(3)-invariant QM9 models | QM9 general-purpose models · QM9 | MAE: better on majority of tasks; e.g. alpha .104 vs .097 for LieConv-SE3 (worse) | Mixed |
| Use LieTransformer (SE2) with canonical lift instead of log map for location attention | Hamiltonian spring dynamics · Hamiltonian spring dynamics | MSE | Helped |
| Use Monte Carlo lift samples: increase number of lift samples for LieTransformer-SE2 (ablation) | shape counting SE(2) equivariance error · Shape counting | invariance error: error ~1e-6 with 3 lift samples, decreasing monotonically | Helped |
| Use LieTransformer instead of LieConv for Hamiltonian dynamics (T(2) and SE(2) models) | Hamiltonian spring dynamics · Hamiltonian spring dynamics | test MSE | Helped |
| Use SE(2) invariance vs T(2) invariance for LieTransformer on shape counting with SE(2) augmentation | LieTransformer on shape counting · Shape counting | test accuracy: T2 0.75 vs SE2 0.71 on D_test; SE2 better on SE2-augmented test (0.69 vs 0.63) | Mixed |
Closest papers by meaning
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.92)
- Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data (ICML 2020 · similarity 0.91)
- A General Framework for Equivariant Neural Networks on Reductive Lie Groups (NeurIPS 2023 · similarity 0.89)
- Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups (ICLR 2025 · similarity 0.89)
- Equivariant Neural Tangent Kernels (ICML 2025 · similarity 0.86)
- Affine Equivariant Networks Based on Differential Invariants (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.