Research map / Large language models / Language, safety and interpretability
The Geometry of Categorical and Hierarchical Concepts in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends the linear representation hypothesis from binary contrast directions to vector representations of binary features, showing categorical concepts are polytopes and hierarchical relations are encoded as orthogonality, validated on Gemma-2B and LLaMA-3-8B with WordNet concepts.
Problem
Prior formalizations of linear representations only handle binary concepts with natural counterfactual contrasts; it is unclear how binary features, categorical concepts, and hierarchical relations between concepts are geometrically encoded in LLM representation spaces.
Approach
Extend the linear representation definition so that binary features get magnitude-carrying vector representations (via the unembedding whitening and a causal inner product), which makes categorical concepts polytopes and yields hierarchical orthogonality theorems (child-parent difference orthogonal to parent vector).
Key results
- Test-word projections onto estimated WordNet feature vectors are near 1 and random words near 0 for original unembeddings, while shuffled unembeddings give near 0 for test words.
- Adding the parent contrast vector changes the parent logit difference by about 5.1 to 14.4 while child logit differences change by under 0.4 in absolute value (Gemma-2B intervention table).
- Cosine similarity between child-parent difference and parent vectors is close to 0 across WordNet noun features, while random-parent controls are far from 0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the normalized plant-to-animal parent contrast representation | Gemma-2B context embeddings · WordNet noun hierarchy: plant.n.02 → animal.n.01 | change in logit differences: 5.1265 ± 1.1731 | Helped |
| Add the normalized plant-to-animal parent contrast representation | Gemma-2B context embeddings · WordNet noun hierarchy: mammal.n.01 → reptile.n.01 | change in logit differences: -0.0600 ± 1.2190 | No clear effect |
| Add the normalized fluid-to-solid parent contrast representation | Gemma-2B context embeddings · WordNet noun hierarchy: fluid.n.02 → solid.n.01 | change in logit differences: 9.8296 ± 1.1099 | Helped |
| Add the normalized fluid-to-solid parent contrast representation | Gemma-2B context embeddings · WordNet noun hierarchy: crystal.n.01 → food.n.02 | change in logit differences: 0.3770 ± 1.5410 | No clear effect |
| Add the normalized scientist-to-contestant parent contrast representation | Gemma-2B context embeddings · WordNet noun hierarchy: scientist.n.01 → contestant.n.01 | change in logit differences: 14.4222 ± 0.9458 | Helped |
| Add the normalized scientist-to-contestant parent contrast representation | Gemma-2B context embeddings · WordNet noun hierarchy: athlete.n.01 → player.n.01 | change in logit differences: -0.1545 ± 1.1426 | No clear effect |
| Use shuffled unembeddings instead of original unembeddings | Gemma-2B vector representation estimation · WordNet noun hierarchy | test-word projection | Hurt |
| Use shuffled unembeddings instead of original unembeddings | Gemma-2B vector representation estimation · WordNet noun hierarchy, sibling relationships | cosine similarity reflecting semantic structure | Hurt |
| Use the naive Euclidean inner product instead of the causal inner product | Gemma-2B representation geometry · WordNet noun hierarchy | hierarchical orthogonality | Hurt |
Surprising findings
- Hierarchical orthogonality also appears with shuffled unembeddings when set inclusion holds, which the authors attribute to set inclusion in estimation; it breaks when train/test splitting removes inclusion.
- Hierarchical orthogonality fails under the naive Euclidean inner product without whitening.
Limitations
- Theory assumes perfect linear representations; in practice representations are only approximately isotropic around the predicted values.
- Relies on the final-layer unembedding whitening; the approach does not extend to internal layers as is.
- Hyperbolic geometry connection is not found in LLMs; concept sets are built from WordNet and ChatGPT-generated tokens.
Experimental setup
Datasets: WordNet noun and verb hierarchy Gemma unembedding vocabulary
Benchmarks: WordNet noun hierarchy (593 synsets) WordNet verb hierarchy (364 synsets)
Metrics: projection of test vs random words onto concept vector change in logit differences after intervention cosine similarity between concept vectors
Baselines: random parent vector shuffled unembeddings Euclidean inner product
Builds on: linear representation hypothesis (Park et al. 2024 causal inner product) WordNet
Closest papers by meaning
- The Linear Representation Hypothesis and the Geometry of Large Language Models (ICML 2024 · similarity 0.92)
- On the Origins of Linear Representations in Large Language Models (ICML 2024 · similarity 0.90)
- From Causal to Concept-Based Representation Learning (NeurIPS 2024 · similarity 0.87)
- Invariant Features in Language Models: Geometric Characterization and Model Attribution (NeurIPS 2026 · similarity 0.86)
- ICLR: In-Context Learning of Representations (ICLR 2025 · similarity 0.86)
- By Tying Embeddings You Are Assuming the Distributional Hypothesis (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.