Research map / Architectures and efficiency / CNN design

A Characterization Theorem for Equivariant Networks with Point-wise Activations

ICLR 2024 · Architectures and efficiency › CNN design · approach: Group symmetry equivariant networks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Gives a complete characterization of exactly equivariant networks with continuous point-wise activations: five maximal dual pairs of activation classes and matrix groups, with consequences for compact/connected groups (rotation-equivariant nets are invariant), disentangled representations, and permutation-equivariant IGNs.

Problem

Which finite-dimensional group representations and coordinate choices are compatible with point-wise activations (e.g., ReLU) so that the resulting layer is exactly equivariant, generalizing the prior partial characterization limited to finite groups and restricted activations.

Approach

Treat activation classes F and matrix groups M as dual objects via maximal-admissible operators M(.) and F(.), show these stabilize after two iterations, and classify the finitely many maximal dual pairs (continuous/permutation, odd/signed permutation, semilinear/non-negative monomial, b- and ±b-multiplicative/monomial). Apply to compact groups (isomorphic to signed permutation reps), rotations, d

Key results

• Theorem gives exactly five dual pairs of maximal admissible activation families and matrix groups (continuous/permutation, odd/signed permutation, semilinear/non-negative monomial, b- and ±b-multiplicative/monomial).
• For compact groups, any representation is isomorphic to a subgroup of (signed) permutation matrices, so continuous odd activations can always be used after a change of basis.
• For connected compact groups such as SO(n), every admissible equivariant representation is trivial, so rotation-equivariant networks with point-wise activations are invariant.
• Disentangled equivariant networks admit point-wise activations if and only if their feature representation is a sum of trivial representations (finite groups).

Measured effects reported in the paper

Surprising findings

• Rotation-equivariant networks with point-wise (non-affine) activations can only be invariant, limiting their use for segmentation or detection.
• Disentangled steerable CNN feature spaces with point-wise activations reduce to trivial representations, and the first-layer linear map from the standard S_n representation to such spaces is zero by Schur's lemma.

Limitations

• Applies only to exact equivariance with finite-dimensional representations; infinite-dimensional harmonic-analysis models are not covered.
• Assumes continuous activations and that the activation class is not purely affine.
• Results on rotation invariance and disentangled trivial representations are negative, so they constrain designs rather than offering positive architectures.

Experimental setup

Baselines: Ravanbakhsh et al. (2017) characterization

Builds on: Ravanbakhsh et al. equivariant network characterization for point-wise activatio Invariant Graph Networks steerable CNNs

Closest papers by meaning

• Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks? (NeurIPS 2024 · similarity 0.91)
• On Universality Classes of Equivariant Networks (NeurIPS 2025 · similarity 0.91)
• A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.91)
• Universally Invariant Learning in Equivariant GNNs (NeurIPS 2025 · similarity 0.90)
• A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs (ICML 2024 · similarity 0.90)
• Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.90)