Research map / Large language models / Language, safety and interpretability
Transformer Block Coupling and its Correlation with Generalization in LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that top singular vectors of transformer block Jacobians become coupling across depth and tokens during training, and that coupling correlates with LLM benchmark performance more strongly than parameter count, depth, or embedding dimension.
Problem
Understanding the internal mechanisms of transformers and whether structural properties of their block-wise linearized dynamics relate to generalization in LLMs.
Approach
Measures coupling between top-K singular vectors of per-block Jacobians across layers and tokens (a coupling matrix and mis-coupling score), and relates it to HuggingFace Open LLM Leaderboard scores and training dynamics; also measures trajectory linearity (line-shape score) and exponential spacing (expodistance).
Key results
- Average coupling across depth and tokens correlates with Open LLM Leaderboard scores across LLMs with R^2 about 0.8 and p-value 9.99e-10, stronger than parameter count, depth, or embedding dimension.
- In ViTs trained on CIFAR-10, coupling correlates with test accuracy with R^2 = 0.93.
- Mean LSS is about 4.25 in trained models versus 6.54 at initialization, indicating increased linearity with training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase average transformer block coupling | LLMs evaluated on Open LLM Leaderboard benchmarks · Open LLM Leaderboard benchmarks | benchmark score: R^2 value of $0.8$; p-value of $9.99\times 10^{-10}$ | Helped |
| Increase coupling during training | Pythia 6.9B and Pythia 12B · Training checkpoints | coupling | Helped |
| Train the model | LLMs · HuggingFace Open LLM Leaderboard prompts | line-shape score (LSS): 6.54 to 4.25 | Helped |
| Increase coupling during training | Vision Transformers trained on CIFAR10 · CIFAR10 test | accuracy: R^2 = 0.93 | Helped |
| Append “Let's think step by step.” to GSM8K prompts | LLaMA models · GSM8K prompts | depth-wise coupling | Helped |
| Append “Let's think step by step.” to GSM8K prompts | Gemma models · GSM8K prompts | depth-wise coupling | No clear effect |
| Append “Let's think step by step.” to GSM8K prompts | Phi-2 · GSM8K prompts | depth-wise coupling | Hurt |
| Append “Let's think step by step.” to GSM8K prompts | Phi-1.5 · GSM8K prompts | depth-wise coupling | Helped |
Surprising findings
- Coupling is measured on training data for ViTs yet still correlates with test accuracy, suggesting a structural property tied to generalization.
- Top singular values of trained LLM block Jacobians do not show a consistent depth scaling across models, unlike classification ResNets.
Limitations
- Correlation across a limited set of LLMs does not establish causality between coupling and generalization.
- Jacobian and coupling computation requires substantial memory, so some models were quantized and prompts were subsampled.
- Theoretical exponential-growth motivation relies on assumptions such as perfect coupling and constant singular values across depth.
Experimental setup
Datasets: ARC GSM8K HellaSwag MMLU TruthfulQA WinoGrande CIFAR-10 WikiText-2
Benchmarks: HuggingFace Open LLM Leaderboard
Metrics: coupling score c_K R^2 p-value line-shape score expodistance CIFAR-10 test accuracy
Baselines: ResNet residual Jacobian analysis
Builds on: Residual Alignment in ResNets Line-shape score trajectories Neural ODEs DEiT
Closest papers by meaning
- Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models (NeurIPS 2025 · similarity 0.91)
- Demystifying Singular Defects in Large Language Models (ICML 2025 · similarity 0.90)
- Building on Efficient Foundations: Effective Training of LLMs with Structured Feedforward Layers (NeurIPS 2024 · similarity 0.90)
- Efficient Large Language Model Inference with Neural Block Linearization (NeurIPS 2025 · similarity 0.89)
- The Role of Sparsity for Length Generalization in LLMs (ICML 2025 · similarity 0.89)
- Understanding Transformers via N-Gram Statistics (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.