Research map / Learning theory and algorithms / Prediction and decision losses
The Linear Representation Hypothesis and the Geometry of Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes the linear representation hypothesis via counterfactual pairs in unembedding and embedding spaces, links them to probing and steering, and derives a causal inner product (Cov(γ)^{-1}) that unifies both representations, validated on LLaMA-2-7B.
Problem
What "linear representation" of high-level concepts in language models formally means, and which inner product makes geometric notions like cosine similarity and projection meaningful in representation space.
Approach
Define unembedding and embedding linear representations via counterfactual pairs, prove the unembedding one gives logit-linear measurement and the embedding one gives intervention, and introduce a causal inner product (orthogonality for causally separable concepts) estimated as Cov(γ)^{-1} under an uncorrelatedness assumption, which makes embedding and unembedding representations coincide.
Key results
- Counterfactual pair difference vectors align with the estimated concept direction in LLaMA-2-7B for most of 27 concepts; thing->part is the exception.
- Adding the intervention representation alpha*lambda_bar_W (alpha up to 0.4) to context representations raised P(queen) to top-1 for 'Long live the', while off-target concept logits stayed roughly constant.
- Under the estimated causal inner product, causally separable concept pairs are near-orthogonal; the Euclidean inner product also looks approximately orthogonal for LLaMA-2 but fails to capture semantics for Gemma-2B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the estimated causal inner product instead of the Euclidean inner product | LLaMA-2-7B concept representations · 27 concepts | semantic alignment of concept directions | Helped |
| Use the estimated causal inner product instead of the Euclidean inner product | Gemma-2B concept representations · 27 concepts | semantic alignment of concept directions | Helped |
| Estimate a concept direction from counterfactual word pairs | LLaMA-2-7B unembedding space · 27 concepts | alignment of counterfactual pair differences | Helped |
| Represent thing/part with a linear direction estimated from counterfactual pairs | LLaMA-2-7B unembedding space · thing/part | alignment of counterfactual pair differences | No clear effect |
| Use the French/Spanish concept direction as a linear probe | LLaMA-2-7B · French and Spanish Wikipedia contexts | concept prediction | Helped |
| Use the male/female direction to predict French versus Spanish | LLaMA-2-7B · French and Spanish Wikipedia contexts | off-target concept prediction | No clear effect |
| Add the male/female intervention representation to the context | LLaMA-2-7B · king/queen and King/king completions | logit for queen versus king | Helped |
| Add the male/female intervention representation to the context | LLaMA-2-7B · king/queen and King/king completions | logit for King versus king | No clear effect |
| Increase the male/female intervention scale from 0 to 0.4 | LLaMA-2-7B · Long live the | rank of queen: 4 -> 1 | Helped |
| Increase the male/female intervention scale from 0 to 0.4 | LLaMA-2-7B · The prince matured and eventually became the | rank of queen: 5 -> 1 | Helped |
| Increase the male/female intervention scale from 0 to 0.4 | LLaMA-2-7B · In a monarchy, the ruler is usually a | rank of queen: 5 -> 2 | Helped |
| Use the Euclidean inner product instead of the estimated causal inner product | LLaMA-2-7B concept representations · 27 concepts | orthogonality of causally separable concepts | No clear effect |
Surprising findings
- The Euclidean inner product is approximately orthogonal for causally separable concepts in LLaMA-2-7B, though it fails for Gemma-2B.
- The concept thing->part does not appear to have a linear representation in the unembedding space.
Limitations
- Causal inner product is identified only up to a free diagonal matrix D; the choice D=I is not principled.
- Relies on an uncorrelatedness assumption for randomly sampled vocabulary unembeddings, which may not hold for natural-text correlations.
- Counterfactual word pairs suffer from tokenization noise (single-token constraint, words embedded in other tokens).
Experimental setup
Datasets: BATS 3.0 Wikipedia contexts word2word bilingual lexicon world capitals csv ChatGPT-4 generated counterfactual pairs and contexts
Benchmarks: BATS 3.0 Wikipedia French/Spanish contexts
Metrics: projection histograms of counterfactual difference vectors inner product heatmaps logit change under intervention correlation of projected unembeddings
Baselines: Euclidean inner product
Builds on: word2vec analogy geometry causal representation learning (diffusion concept vectors) Riesz isomorphism
Closest papers by meaning
- The Geometry of Categorical and Hierarchical Concepts in Large Language Models (ICLR 2025 · similarity 0.92)
- On the Origins of Linear Representations in Large Language Models (ICML 2024 · similarity 0.88)
- Invariant Features in Language Models: Geometric Characterization and Model Attribution (NeurIPS 2026 · similarity 0.86)
- From Causal to Concept-Based Representation Learning (NeurIPS 2024 · similarity 0.86)
- On Linear Representations and Pretraining Data Frequency in Language Models (ICLR 2025 · similarity 0.86)
- How are linear representations learned? Exact solutions to the dynamics of abstraction (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.