Research map / Architectures and efficiency / RNNs and normalization

Not All Language Model Features Are One-Dimensionally Linear

ICLR 2025 · Architectures and efficiency › RNNs and normalization · approach: Layer-wise analysis of language models · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formalizes irreducible multi-dimensional features, finds circular features (days, months) in GPT-2 and Mistral 7B via clustering sparse autoencoder dictionary elements, and shows causal use of these circles in modular-arithmetic-like calendar tasks.

Problem

Language model features are assumed to be one-dimensional linear directions; the paper asks whether some representations are inherently multi-dimensional and how to find them automatically.

Approach

Defines irreducibility of multi-dimensional features via statistical separability and mixture conditions, proposes a multi-dimensional superposition hypothesis, and uses clustering of SAE dictionary elements plus PCA-based tests to automatically discover circular features that are then causally validated by subspace patching.

Key results

• Clustering SAE dictionary elements in GPT-2 and Mistral 7B recovers interpretable circular features for days of the week and months of the year; the cluster ranks 9, 28, and 15 among 1000 clusters by the combined irreducibility score.
• Subspace patching on the circular subspace in Mistral 7B and Llama 3 8B causally affects the output logits, with early-layer circle interventions nearly matching full-layer patching.
• An SAE-derived plane probe on Mistral layer 8 gives average logit difference -2.01 versus -2.58 for a trained circular probe, and is more robust to layer shifts (-2.32 vs 0.029 on layer 6).

Measured effects reported in the paper

Surprising findings

• Mistral 7B and Llama 3 8B get trivial accuracy on plain modular addition prompts (e.g. 5 + 3 mod 7) despite the calendar tasks being equivalent to modular arithmetic.
• GPT-2 has circular representations but gets trivial accuracy on the Weekdays and Months tasks.

Limitations

• Definitions of irreducibility are purely statistical rather than intervention-based and had to be relaxed to practical degrees, giving subjective measures.
• Only a small number of interpretable multi-dimensional features were found, and it is unclear why more were not found.
• Circular interventions were tested only on specific calendar prompts and a limited set of models.

Experimental setup

Datasets: The Pile Alpaca synthetic circle data Weekdays and Months prompts

Benchmarks: Weekdays task prompts Months task prompts

Metrics: average logit difference separability index epsilon-mixture index accuracy (highest logit valid token)

Baselines: circular probe on PCA projections top-5 PCA patching full layer patching

Builds on: sparse autoencoders linear representation hypothesis superposition hypothesis activation patching

Closest papers by meaning

• Emergence of a High-Dimensional Abstraction Phase in Language Transformers (ICLR 2025 · similarity 0.86)
• Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (ICLR 2025 · similarity 0.85)
• Correlation Dimension of Autoregressive Large Language Models (NeurIPS 2025 · similarity 0.85)
• How do Large Language Models Handle Multilingualism? (NeurIPS 2024 · similarity 0.85)
• On the Origins of Linear Representations in Large Language Models (ICML 2024 · similarity 0.85)
• Geometry of Decision Making in Language Models (NeurIPS 2025 · similarity 0.84)