Research map / Learning theory and algorithms / Kernels and regression theory
A Canonicalization Perspective on Invariant and Equivariant Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reframes frame averaging as canonicalization, proves frame–canonicalization equivalence, resolves SignNet universality (it is not universal), and proposes OAP eigenvector canonicalizations that are optimal (no permutation) or superior to MAP/FA-lap (with permutation).
Problem
Frame averaging lacks a principled way to characterize frame complexity and optimality, and open questions remain on the expressivity of sign/basis-invariant eigenvector networks such as SignNet and BasisNet.
Approach
Replace frames with set-valued orbit canonicalizations (averaging over canonical forms rather than group elements), showing frames and canonicalizations are equivalent with complexity |C(X)|=|F(X)|/|G_X|; use this to design OAP (orthogonalized axis projection) eigenvector canonicalizations and to prove SignNet is not universal.
Key results
- OAP-graph frames/canonicalizations are about 82x and 77x smaller than FA-graph on EXP (avg |F| 2.55e22 vs 2.10e24).
- On ZINC, OAP reduces non-canonicalizable basis eigenvectors to 0.2% versus 1.0% for MAP and 2.2% for FA-lap.
- With GatedGCN on ZINC, OAP reaches MSE 0.118 vs 0.121 for SignNet and 0.120 for MAP (500K budget); OAP+LSPE reaches 0.098.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply canonical averaging to GIN with node IDs | GIN with node IDs on Exp · Exp | accuracy: 50% -> 100% | Helped |
| Replace FA-graph with OAP-graph | Graph canonicalization on Exp · Exp | average canonicalization size: $5.84\times 10^{21}$ -> $7.57\times 10^{19}$ | Helped |
| Use OAP instead of MAP for basis canonicalization | Eigenvectors on ZINC · ZINC | non-canonicalizable eigenvector ratio, basis: 1.0\% -> \textbf{0.2\%} | Helped |
| Use OAP LapPE instead of MAP LapPE | GatedGCN · ZINC | MSE: 0.120 ± 0.005 -> \textbf{0.118 ± 0.0006} | Helped |
| Add LSPE layers to OAP (ablation) | GatedGCN + OAP · ZINC | MSE: \textbf{0.118 ± 0.0006} -> \textbf{0.098 ± 0.0009} | Helped |
| Use OAP LapPE instead of MAP LapPE | PNA · ZINC | MSE: 0.101 ± 0.005 -> \textbf{0.098 ± 0.0008} | Helped |
| Add LSPE layers to OAP (ablation) | PNA + OAP · ZINC | MSE: \textbf{0.098 ± 0.0008} -> \textbf{0.095 ± 0.004} | Helped |
| Use OAP LapPE instead of MAP LapPE | GatedGCN · MOLTOX21 | ROCAUC: 0.784 ± 0.005 -> \textbf{0.787 ± 0.005} | Helped |
| Use OAP LapPE instead of MAP LapPE | PNA · MOLTOX21 | ROCAUC: 0.761 ± 0.002 -> 0.768 ± 0.002 | Helped |
| Use OAP LapPE instead of MAP LapPE | GatedGCN · MOLPCBA | AP: 0.268 ± 0.002 -> \textbf{0.270 ± 0.002} | Helped |
| Use OAP LapPE instead of MAP LapPE | PNA · MOLPCBA | AP: \textbf{0.281 ± 0.003} -> 0.279 ± 0.002 | Mixed |
Surprising findings
- SignNet and BasisNet with first-order permutation-equivariant backbones are not universal approximators of permutation-equivariant sign/basis-invariant functions, resolving an open problem.
- Frame averaging with permutation-breaking frames performs worst on ZINC, below no positional encoding baseline in some settings.
Limitations
- Optimality of the permutation-equivariant OAP for Laplacian eigenvectors remains unresolved.
- Frame sizes on EXP remain very large, so sampling is still needed.
- Canonicalization-based gains on molecular benchmarks are modest and computational overhead of SignNet is higher.
Experimental setup
Datasets: EXP ZINC OGBG-MOLTOX21 OGBG-MOLPCBA n-body
Benchmarks: EXP ZINC MOLTOX21 MOLPCBA
Metrics: accuracy MSE ROCAUC AP frame/canonicalization size non-canonicalizable ratio
Baselines: SignNet MAP FA-lap LapPE + random sign GCN GAT GIN ChebNet FA-GIN+ID
Builds on: frame averaging MAP canonicalization SignNet BasisNet Laplacian canonicalization
Closest papers by meaning
- Equivariant Frames and the Impossibility of Continuous Canonicalization (ICML 2024 · similarity 0.91)
- Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency (ICML 2024 · similarity 0.90)
- Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.88)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.88)
- Learning (Approximately) Equivariant Networks via Constrained Optimization (NeurIPS 2025 · similarity 0.87)
- Learning symmetries via weight-sharing with doubly stochastic tensors (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.