Research map / Architectures and efficiency / RNNs and normalization
On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders
Elana Simon, Etowah Adams, James Zou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| initialize encoder bias with the activation mean (mean-centering), dict size 2048 (ablation) | TopK SAE · ESM3 layer 24 SwissProt concepts (k=16) | concepts captured: 61 -> 100 | Improved |
| mean-centering, dict size 4096 (ablation) | TopK SAE · ESM3 layer 24 SwissProt concepts (k=16) | concepts captured: 69 -> 121 | Improved |
| mean-centering, dict size 8192 (ablation) | TopK SAE · ESM3 layer 24 SwissProt concepts (k=16) | concepts captured: 73 -> 127 | Improved |
| mean-centering instead of 4x larger dictionary | TopK SAE · ESM3 layer 24 SwissProt concepts (k=16) | concepts captured: 73 -> 100 | Improved |
| mean-centering on synthetic data (ablation) | TopK SAE · Synthetic data (gamma=40) | MMCS with ground truth: 0.38 -> 0.97 | Improved |
| mean-centering on ESM3 layer 24 dead features (ablation) | TopK SAE · ESM3 layer 24 (gamma~8) | dead features (%): 75% -> near zero | Improved |
| add AuxK on ESM3 layer 24 (ablation) | TopK SAE · ESM3 layer 24 (gamma~8) | dead features (%): 75% -> 25% | Improved |
| freeze the bias during training (ablation) | TopK SAE · Synthetic activations (gamma=20) | dead-by-ReLU recovery: permanently dead | Hurt |
| freeze the bias during training (ablation) | TopK SAE · Synthetic activations (gamma=20) | dead-by-TopK recovery | No clear effect |
| add AuxK loss (ablation) | TopK SAE · Synthetic activations (gamma=20) | bias convergence speed | No clear effect |
| mean-centering on Evo1 (extremely low-rank activations) (ablation) | TopK SAE · Evo1 | dead features (%): 73% dead at initialization, ~58% after training | Hurt |
| apply PCA whitening after centering (ablation) | TopK SAE · Evo1 and other low-rank layers | dead features (%): 0% | Improved |
Closest papers by meaning
- Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines? (NeurIPS 2026 · similarity 0.88)
- Scaling and evaluating sparse autoencoders (ICLR 2025 · similarity 0.87)
- Understanding and Minimising Outlier Features in Transformer Training (NeurIPS 2024 · similarity 0.87)
- Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders (NeurIPS 2024 · similarity 0.86)
- Sparse Autoencoders Do Not Find Canonical Units of Analysis (ICLR 2025 · similarity 0.86)
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
