Research map / Architectures and efficiency / Attention and transformers

Transformers Converge to Invariant Algorithmic Cores

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: Transformer expressivity and learnability · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Algorithmic Core Extraction (ACE), which finds activation subspaces that are both active and output-relevant and validates them causally, then applies it to Markov-chain transformers, modular-addition grokking models, and six pretrained LMs to show shared low-dimensional cores.

Problem

Trained transformers are underdetermined: many weight configurations implement the same function, so mechanistic explanations of a single run may reflect training accidents rather than the computation itself.

Approach

ACE extracts the SVD of the hidden-activation/Jacobian interaction to get a low-dimensional subspace that is jointly active and output-relevant, validates it via necessity/sufficiency ablations, and fits linear operators inside it; the paper frames cores as invariants across independent realizations and links weight decay redundancy to grokking time.

Key results

• Markov-chain transformers' 3D cores are necessary and sufficient under ablation (core-only 0.748 = full model; core-removed ~0.24-0.26 vs chance) and recover the chain's nontrivial eigenvalues within a few percent despite projector overlaps of 0.02-0.04.
• In six LMs, a single 1D subject-verb agreement core yields core-only AUC 0.894-0.997 and core-flipped AUC 0.021-0.092, with cross-model Spearman correlations ranging from 0.59 to 0.93.
• Modular-addition cores inflate from about 15 to 60 dimensions under continued weight decay, while sufficiency dimensions stay stable.

Surprising findings

• Weight decay inflates cores with redundant rotational modes after grokking rather than compressing them.
• Steering a single core axis inverts grammatical number throughout open-ended generation, with errors generalizing beyond the defining verbs.

Limitations

• Multi-step reasoning tasks are untested; cores may not stay low-dimensional there.
• Cores are extracted for task-specific behaviors only; systematic task decomposition for multifunctional models is open.
• The grokking scaling law is a heuristic ODE model with empirically fitted constants (Omega, p_crit).

Experimental setup

Datasets: synthetic four-state Markov chain sequences modular addition mod 53 pairs subject-verb agreement prompts (1,200 constructed)

Benchmarks: Markov chain next-token accuracy modular addition test accuracy subject-verb agreement AUC

Metrics: accuracy AUC R^2 (oracle-normalized) CCA Spearman correlation Pearson correlation eigenvalue match projector overlap

Baselines: Bayes-optimal predictor chance accuracy oracle R^2 PCA

Builds on: balanced truncation Kalman decomposition canonical correlation analysis grokking

Closest papers by meaning

• From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers (NeurIPS 2025 · similarity 0.89)
• How Transformers Learn Structured Data: Insights From Hierarchical Filtering (ICML 2025 · similarity 0.88)
• Algorithmic Capabilities of Random Transformers (NeurIPS 2024 · similarity 0.88)
• Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learning (NeurIPS 2026 · similarity 0.88)
• From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics (NeurIPS 2025 · similarity 0.88)
• Selective Induction Heads: How Transformers Select Causal Structures in Context (ICLR 2025 · similarity 0.87)