Research map / Label-efficient and robust learning / Recognition and long-tail
Singular Vectors of Attention Heads Align with Features
Gabriel Franco, Carson Loughridge, Mark Crovella
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add an attention head trained on a feature pair to the toy autoencoder | toy model (20 features, D=10) · toy model | singular vector-feature alignment | Improved |
| increase feature anisotropy up to the range seen in GPT-2 (ablation) | toy model with frozen features · anisotropy sweep | mean cosine similarity: mean cosine similarity above 0.75 | No clear effect |
| anti-whiten singular vectors using the feature covariance (ablation) | SVF alignment under anisotropy · anisotropy sweep | alignment (avg cos sim) | Improved |
| randomly rotate U and V singular vector matrices (keeping spectrum) (ablation) | Pythia-160M IOI heads · sparse attention decomposition, S(v) | decomposition sparseness | Hurt |
| train the model longer (checkpoints) | Pythia-160M IOI circuit heads · IOI heads | number of singular vectors to reconstruct relative attention: typically only 1-4 singular vectors needed | Improved |
| present feature pairs of interest in tokens vs absent (ablation) | toy model attention decomposition · toy model | decomposition sparseness | Improved |
| add rotary positional encoding to the toy model (ablation) | toy model · toy model with RoPE | SVF alignment | No clear effect |
| have more features of interest than head dimension (ablation) | toy model · over-capacity regime | SVF alignment | Mixed |
Closest papers by meaning
- Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (NeurIPS 2025 · similarity 0.87)
- Invariant Features in Language Models: Geometric Characterization and Model Attribution (NeurIPS 2026 · similarity 0.87)
- Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features (ICML 2024 · similarity 0.86)
- Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach (CVPR 2025 · similarity 0.85)
- Attention Alignment Between Humans and Vision-Language Models (NeurIPS 2026 · similarity 0.85)
- Mind the Heads: Topological Representation Alignment for Multimodal LLMs (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
