Research map / Architectures and efficiency / RNNs and normalization
Grokking Group Multiplication with Cosets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reverse engineers one-hidden-layer MLPs that grokked multiplication in S5 and S6, finding they implement coset circuits based on conjugate subgroups rather than the GCR irrep algorithm claimed by prior work.
Problem
How do small one-hidden-layer networks that grok permutation group multiplication (S5, S6) internally implement the task, and do prior mechanistic interpretations of the same models hold up?
Approach
Shows that neurons' pre-activations are nearly constant on cosets of subgroups (measured by a coset-concentration statistic C_H), and that the model implements group multiplication by picking out shared cosets of conjugate subgroups; validates this with ablations and causal interventions and argues against the GCR explanation.
Key results
- Over 99.2% of linear-layer neurons in S5 and S6 models had min coset concentration below 1.0, most below 1e-6.
- Removing neurons with coset concentration >= 1 left accuracy unchanged in 126 of 128 S5 models.
- Switching left and right embeddings dropped S5 accuracy to 1% on average, while replacing ReLU with absolute value kept 100% accuracy and reduced loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| swap the left and right embeddings (ablation) | One-hidden-layer fully connected network · S5 | Mean Accuracy: 99.99% -> 1% | Hurt |
| switch the signs of both left and right embeddings (ablation) | One-hidden-layer fully connected network · S5 | Mean Accuracy: 99.99% -> 100% | No clear effect |
| switch the sign of the left embedding only (ablation) | One-hidden-layer fully connected network · S5 | Mean Accuracy: 99.99% -> 0% | Hurt |
| switch the sign of the right embedding only (ablation) | One-hidden-layer fully connected network · S5 | Mean Accuracy: 99.99% -> 0% | Hurt |
| add noise from N(0, 0.1) (ablation) | One-hidden-layer fully connected network · S5 | Mean Accuracy: 99.99% -> 99.99% | No clear effect |
| add noise from N(0, 1) (ablation) | One-hidden-layer fully connected network · S5 | Mean Accuracy: 99.99% -> 97.8% | Hurt |
| replace ReLU with absolute value (ablation) | One-hidden-layer fully connected network · S5 | Mean Loss: 1.97e-6 -> 3.69e-13 | Helped |
| add noise from N(1, 1) (ablation) | One-hidden-layer fully connected network · S5 | Mean Accuracy: 99.99% -> 88% | Hurt |
| add noise from N(-1, 1) (ablation) | One-hidden-layer fully connected network · S5 | Mean Accuracy: 99.99% -> 98% | Hurt |
| remove neurons with coset concentration min C_H(N_i) >= 1 (ablation) | S5 coset circuit models · S5 | Accuracy: 126 models saw no change; two decreased to 99% and 98% | No clear effect |
| remove neurons with coset concentration min C_H(N_i) >= 10^-5 (ablation) | S5 coset circuit models · S5 | Accuracy: 50th and 25th percentile accuracy is still 100% | No clear effect |
| remove neurons with coset concentration min C_H(N_i) >= 10^-7 (ablation) | S5 coset circuit models · S5 | Accuracy: median accuracy is still 100% | Mixed |
Surprising findings
- Replacing ReLU with absolute value preserved perfect accuracy and roughly halved the loss, which the GCR account does not predict.
- The same trained models were interpreted differently by the prior GCR work, showing how easily interpretability conclusions diverge.
Limitations
- Analysis limited to a toy one-hidden-layer model on small permutation groups, which may not transfer to real-world models.
- Evidence is largely observational for the GCR comparison, and the authors note the GCR evidence was not fully replicated.
Experimental setup
Datasets: S5 group multiplication table (40% train split) S6 group multiplication table (40% train split)
Benchmarks: S5 multiplication accuracy S6 multiplication accuracy
Metrics: accuracy validation loss coset concentration C_H causal intervention accuracy and loss
Baselines: Group Composition via Representations (GCR) algorithm
Builds on: A Toy Model of Universality Grokking Group Composition via Representations
Closest papers by meaning
- Towards a Unified and Verified Understanding of Group-Operation Networks (ICLR 2025 · similarity 0.87)
- Why Do You Grok? A Theoretical Analysis on Grokking Modular Addition (ICML 2024 · similarity 0.84)
- To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.84)
- Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks (NeurIPS 2025 · similarity 0.84)
- Flatness is Necessary, Neural Collapse is Not: Rethinking Generalization via Grokking (NeurIPS 2025 · similarity 0.83)
- Emergence in non-neural models: grokking modular arithmetic via average gradient outer product (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.