Research map / Architectures and efficiency / RNNs and normalization
Sparse Autoencoders Do Not Find Canonical Units of Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that sparse autoencoders of different widths are neither complete nor atomic, using SAE stitching to find novel vs. reconstruction latents and meta-SAEs to decompose SAE decoder directions into interpretable meta-latents.
Problem
Whether sparse autoencoders (SAEs) recover a canonical set of units of analysis in LLMs, i.e. a unique, complete, and atomic feature dictionary, and how dictionary size affects the learned latents.
Approach
SAE stitching swaps or inserts latents between SAEs of different widths based on decoder cosine similarity to classify larger-SAE latents as novel (improve reconstruction when added, so smaller SAEs are incomplete) or reconstruction latents (replaceable). Meta-SAEs, trained on the decoder directions of a larger SAE, decompose its latents into sparse combinations of meta-latents, showing they are n
Key results
- Adding novel latents from GPT2-1536 to GPT2-768 and swapping reconstruction latents yields interpolation between SAE sizes, with adding all latents of cosine similarity <=0.7 reducing MSE by almost 10%.
- Meta-SAE (2304 meta-latents) on a 49152-latent GPT-2 SAE explains 55.47% of decoder variance, with on average 4 meta-latents per SAE latent.
- GPT-4o-mini picks the correct SAE latent explanation out of 5 options 73% of the time on 1,000 sampled latents.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase GPT-2 SAE dictionary size from 768 to 98304 | GPT-2 Small residual-stream SAEs · GPT-2 Small layer 8 residual stream | MSE: 2.72 -> 1.14 | Helped |
| Increase Gemma SAE dictionary size from 16384 to 32768 | Gemma Scope SAEs · Gemma 2 2B layer 12 residual stream | MSE: 1.72 -> 1.59 | Helped |
| Add novel latents from GPT2-1536 to GPT2-768 (ablation) | GPT2-768 · GPT-2 Small layer 8 residual stream | MSE: almost 10% | Helped |
| Swap related reconstruction latents from a larger SAE for similar latents in the smaller SAE (ablation) | Smaller GPT-2 SAEs · GPT-2 Small layer 8 residual stream | L0 | Helped |
| Swap related reconstruction latents from a larger SAE for similar latents in the smaller SAE (ablation) | Smaller GPT-2 SAEs · GPT-2 Small layer 8 residual stream | MSE | Hurt |
| Add novel latents from GemmaScope-32768 to GemmaScope-16384 (ablation) | GemmaScope-16384 · Gemma 2 2B layer 12 residual stream | MSE | Helped |
| Replace decoder directions of the meta-SAE with the most similar directions from an SAE of similar dictionary size (ablation) | Meta-SAE trained on GPT2-49152 decoder directions · Meta-SAE reconstruction | reconstruction performance: only a small decrease | Hurt |
| Use meta-SAE explanations to identify the matching SAE latent | GPT-4o-mini evaluating meta-SAE explanations · Random sample of 1,000 SAE latents | multiple-choice accuracy: 73% | Helped |
| Increase GPT-2 SAE dictionary size | GPT-2 Small SAEs · Sparse probing across 8 benchmark datasets | accuracy | Mixed |
| Increase GPT-2 SAE dictionary size | GPT-2 Small SAEs · Targeted Probe Perturbation across 2 benchmark datasets | targeted probe perturbation score | Mixed |
| Exchange decoder bias vectors between SAEs of different dictionary sizes (ablation) | GPT-2 Small SAEs · SAE reconstruction | MSE | No clear effect |
Surprising findings
- Sparse probing and targeted probe perturbation accuracy are not monotonic in SAE dictionary size, with complex dataset-dependent relationships.
Limitations
- Only two LLMs studied (GPT-2 Small and Gemma 2 2B).
- Does not include very large SAEs.
- Methods do not identify canonical units or the correct dictionary size for a task.
Experimental setup
Datasets: OpenWebText SAE decoder directions (meta-SAE training)
Benchmarks: sparse probing targeted probe perturbation
Metrics: MSE L0 cross-entropy loss recovered multiple-choice accuracy of explanations sparse probing accuracy TPP accuracy
Baselines: Gemma Scope SAEs
Builds on: BatchTopK SAE TopK SAE Gemma Scope Towards Monosemanticity model stitching
Closest papers by meaning
- Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines? (NeurIPS 2026 · similarity 0.91)
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.90)
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.90)
- Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry (NeurIPS 2025 · similarity 0.90)
- Learning Multi-Level Features with Matryoshka Sparse Autoencoders (ICML 2025 · similarity 0.89)
- Mechanistic Interpretability with Sparse Autoencoder Neural Operators (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.