Research map / Large language models / Language, safety and interpretability
Emergence of a High-Dimensional Abstraction Phase in Language Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that five pretrained transformer LMs share a high-intrinsic-dimension phase in middle layers (GRIDE estimator) that marks the first full linguistic abstraction, transfers to downstream tasks, aligns across models, and whose earlier onset correlates with better next-token prediction.
Problem
How the geometry of LM layer representations (intrinsic dimension across layers) relates to their function in linguistic processing and next-token prediction quality.
Approach
Layerwise intrinsic dimension (estimated with GRIDE at scale-plateau k) reveals a distinct high-ID expansion phase in the early-to-mid layers of pretrained LMs; this phase coincides with a shift from surface-form to abstract syntactic/semantic information, cross-model representational agreement (Information Imbalance), and earlier onset correlating with lower surprisal.
Key results
- Across five 7B-class LMs, intrinsic dimension peaks at roughly O(10) in early-to-mid layers (e.g., layers ~6-20), with OPT's peak lasting from layer 17 to 23.
- Spearman correlation between surprisal and ID-peak onset is rho=0.65 (p=0.01), so earlier peak onset tracks better next-token prediction; max ID vs surprisal gives rho=-0.46 (p=0.09).
- Shuffled-corpus inputs yield low ID throughout layers and chance-level probing, indicating the peak reflects learned linguistic structure.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| A higher maximum ID across layers | Five trained language models on sane text · (model, corpus) combinations | Average prediction surprisal: ρ=-0.46, p=0.09 | Helped |
| An earlier ID-peak onset | Five trained language models on sane text · (model, corpus) combinations | Average prediction surprisal: ρ=0.65, p=0.01 | Helped |
| Process shuffled rather than original text (ablation) | Five pre-trained language models · Shuffled Bookcorpus, Pile, and WikiText | Intrinsic dimension across layers | Hurt |
| Train the model rather than use an untrained model (ablation) | Pythia · Bookcorpus, Pile, and WikiText | Intrinsic dimension across layers | Helped |
| Use ID-peak representations rather than representations outside the peak | Five pre-trained language models · Syntactic and semantic probing tasks | Accuracy | Helped |
| Use ID-peak representations rather than earlier representations | Five pre-trained language models · Surface-form probing tasks | Accuracy | Hurt |
| Use ID-peak representations rather than earlier representations | Five pre-trained language models · Toxicity and sentiment classification | Validation accuracy | Helped |
| Use shuffled example labels instead of the original labels (ablation) | MLP linguistic probes · Linguistic probing tasks | Accuracy | Hurt |
| Increase Pythia model size | Pythia · WikiText | Peak intrinsic dimension | Helped |
| Double or halve the GRIDE neighbourhood scale within the plateau region (ablation) | Five language models · Bookcorpus, Pile, and WikiText | Intrinsic dimension estimates | No clear effect |
Surprising findings
- Semantic/syntactic probing asymptote and transfer convergence occur within the ID expansion phase, contrary to Valeriani et al. who placed semantic processing in the low-ID valley.
- The ID peak layers of Pythia and OPT directionally contain other models' representations, with cross-model Information Imbalance showing asymmetry.
Limitations
- Correlation between ID onset and surprisal is based on only 50 (model, corpus) points and the max-ID correlation is not significant (p=0.09).
- Analysis is correlational; the authors state causal links require layer ablations and dimension constraints.
- Scale k is chosen by visual plateau inspection per model-corpus combination.
Experimental setup
Datasets: Bookcorpus Pile-10k WikiText-103 SentEval probing tasks IMDb Jigsaw toxicity
Benchmarks: SentEval probing tasks (Sentence Length, Word Content, Bigram Shift, Coordination Inversion, Odd Man Out) IMDb sentiment Jigsaw toxicity
Metrics: intrinsic dimension (GRIDE) information imbalance probing accuracy validation accuracy Spearman correlation with surprisal CKA
Baselines: Valeriani et al. (ID across layers in vision and protein transformers) CKA RSA neighborhood overlap
Builds on: GRIDE TwoNN Information Imbalance SentEval probing tasks
Closest papers by meaning
- Language Models Are Implicitly Continuous (ICLR 2025 · similarity 0.89)
- Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale (NeurIPS 2025 · similarity 0.88)
- The Same but Different: Structural Similarities and Differences in Multilingual Language Modeling (ICLR 2025 · similarity 0.87)
- Less is More: Local Intrinsic Dimensions of Contextual Language Models (NeurIPS 2025 · similarity 0.87)
- Geometry of Decision Making in Language Models (NeurIPS 2025 · similarity 0.87)
- Talking Heads: Understanding Inter-Layer Communication in Transformer Language Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.