Research map / Architectures and efficiency / RNNs and normalization
Positional Encoding Is All You Need For Scalable Equivariance Constraint Relaxation
Hagay Michaeli, Haggai Maron, Daniel Soudry
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes using positional encodings to relax equivariance constraints in neural networks, allowing approximate symmetry while retaining scalability (details limited to the abstract).
Problem
Strict equivariance can hinder optimization when data only approximately satisfy the assumed symmetry or when equivariance constraints create a difficult loss landscape.
Approach
Relaxes strict equivariance constraints via positional encodings rather than fully equivariant architectures; the abstract does not give further specifics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add annealed positional-encoding relaxation (additive) | VN-PointNet · ModelNet40 SO(3)/SO(3) | instance accuracy: 67.5 -> 79.6 | Improved |
| use PE relaxation instead of dense-layer CR | VN-PointNet · ModelNet40 SO(3)/SO(3) | instance accuracy: 74.0 -> 79.6 | Improved |
| add annealed PE relaxation | VN-DGCNN · ModelNet40 SO(3)/SO(3) | instance accuracy: 88.3 -> 88.6 | No clear effect |
| use PE instead of dense relaxation | VN-PointNet · ModelNet40 SO(3)/SO(3) | training time per epoch (s): 97.6 -> 27.9 | Improved |
| add annealed PE relaxation | Rapidash-Eq · ModelNet40 Rapidash | accuracy: 83.7 -> 84.6 | Improved |
| add annealed PE (train only) instead of keeping PE at test | AF-XCiT-Nano · ImageNet shift accuracy | shift accuracy: 70.3 -> 71.1 | Improved |
| add annealed PE relaxation | AF-XCiT-Nano · ImageNet val | top-1 accuracy: 70.6 -> 71.1 | Improved |
| add annealed PE relaxation | E2CNN · Smoke plume (Future) | RMSE x1e-2: 81.2 -> 79.8 | Improved |
| add PE relaxation | SEGNN · N-Body | test MSE x1e-4: 44.3 -> 41.9 | Improved |
| make PE scale gamma learnable (ablation) | VN-PointNet · ModelNet40 SO(3)/SO(3) ablation | accuracy w/o PE: 4.5 -> 79.7 | Improved |
| add linear decay schedule with learnable gamma (ablation) | VN-PointNet · ModelNet40 SO(3)/SO(3) ablation | accuracy w/o PE: 79.7 -> 80.4 | No clear effect |
| add explicit regularization beta=0.001 (learnable gamma) (ablation) | VN-PointNet · ModelNet40 SO(3)/SO(3) ablation | accuracy w/o PE: 79.7 -> 80.7 | No clear effect |
Closest papers by meaning
- Improving Equivariant Model Training via Constraint Relaxation (NeurIPS 2024 · similarity 0.94)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.92)
- Learning (Approximately) Equivariant Networks via Constrained Optimization (NeurIPS 2025 · similarity 0.91)
- Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.90)
- Approximately Equivariant Neural Processes (NeurIPS 2024 · similarity 0.89)
- A Characterization Theorem for Equivariant Networks with Point-wise Activations (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
