Research map / Architectures and efficiency / CNN design
Towards a Unified and Verified Understanding of Group-Operation Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives a mechanistic explanation (the rho-set circuit, a bi-equivariant structure) of one-hidden-layer networks trained on S5 group composition, unifying prior irrep and coset explanations, and verifies it via compact proofs of accuracy lower bounds.
Problem
Prior mechanistic interpretations of one-hidden-layer networks trained on finite group composition disagree (irrep composition vs coset concentration) and are incomplete; there is no rigorous quantitative way to evaluate explanation completeness.
Approach
Models implement rho-set circuits: neurons are projections of irrep actions on permutation-invariant vector sets, making the network bi-equivariant in each input; this explanation is turned into a verifier producing compact proofs of accuracy bounds, with tightness and runtime as completeness metrics.
Key results
- The rho-set verifier yields near-100% accuracy bounds for about 55% of 100 S5 models (mean bound 91.7% on those), while the coset verifier gives 0% for all models.
- Rho-set verifier runs in mean 0.75s versus 2.20s for brute force (about 3x faster).
- Mean accuracy bound over all 100 models is 50.4% for rho-sets, versus 100% for brute force and 0% for coset verifier.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the ρ-set explanation to construct the verifier instead of brute-force enumeration | Accuracy verification for one-hidden-layer neural networks trained on S5 · S5 (100 trained models) | runtime: 2.20s -> 0.75s | Helped |
| Use the ρ-set explanation rather than the coset-concentration explanation | Accuracy verification for one-hidden-layer neural networks trained on S5 · S5 (100 trained models) | mean accuracy lower bound: 0% -> 50.4% | Helped |
| Use the ρ-set explanation rather than brute-force enumeration | Accuracy verification for one-hidden-layer neural networks trained on S5 · S5 (100 trained models) | mean accuracy lower bound: 100% -> 50.4% | Hurt |
| Restrict evaluation to models without va-bad or ρ-bad deviations | ρ-set verifier (Vρ) · S5 | mean accuracy lower bound: 50.4% -> 91.7% | Helped |
| Apply the ρ-set explanation to models without va-bad or ρ-bad deviations | ρ-set verifier (Vρ) · S5 | share of models with accuracy bound ≥95%: ≥95% accuracy bound for 45% of the models we trained | Helped |
| Replace original model parameters with the full idealized ρ-set model (ablation) | Models trained on S5 without va-bad or ρ-bad deviations · S5 | cross-entropy loss | Helped |
Surprising findings
- Coset concentration verifier gives vacuous bounds on every trained model, even though prior work claimed a coset algorithm.
- Irrep ordering of learning frequency tracks minimum rho-set size rather than dimensionality.
Limitations
- Rho-set explanation only covers roughly half of the S5 models; the rest deviate (a-bad or rho-bad cases).
- Results are shown mainly for S5 in the main text; the explanation is validated on a narrow setting.
Experimental setup
Datasets: S5 group multiplication table
Benchmarks: S5 group composition (all 120x120 input pairs)
Metrics: accuracy lower bound verifier runtime bi-equivariance measure
Baselines: brute force verifier coset concentration verifier vacuous verifier
Builds on: irrep sparsity (Chughtai et al.) coset concentration (Stander et al.) compact proofs (Gross et al.) grokking
Closest papers by meaning
- Grokking Group Multiplication with Cosets (ICML 2024 · similarity 0.87)
- MatrixNet: Learning over symmetry groups using learned group representations (NeurIPS 2024 · similarity 0.86)
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.86)
- Monomial Matrix Group Equivariant Neural Functional Networks (NeurIPS 2024 · similarity 0.86)
- Reconsidering Faithfulness in Regular, Self-Explainable and Domain Invariant GNNs (ICLR 2025 · similarity 0.85)
- On Universality Classes of Equivariant Networks (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.