Research map / Large language models / Language, safety and interpretability
At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization
Praneet Suresh, Jack Stanley, Sonia Joseph, Luca Scimeca, Danilo Bzdok
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add typos to 5% of MMLU prompt words | Llama 3.1 8B · MMLU | accuracy: 66.7% -> 51.01% | Hurt |
| add typos to 5% of MMLU prompt words | GPT-4o mini · MMLU | accuracy: 82.10% -> 74.85% | Hurt |
| add typos to 5% of MMLU prompt words | GPT-5-thinking-nano · MMLU | accuracy: 91.26% -> 86.45% | Hurt |
| increase typo rate from clean to every word | GPT-2 25M · TinyStories layer 6 | number of active SAE concepts: nearly 30% more features | Hurt |
| fine-tune on high-energy-score samples instead of lowest decile (ablation) | GPT-2 25M · TinyStories typos (layer 6) | validation loss: over 3% lower final loss | Improved |
| select fine-tuning samples by SAE energy score instead of entropy or Mahalanobis distance (ablation) | GPT-2 25M · TinyStories typos | validation loss: lower final loss at every decile | Improved |
| align SAE features of successful jailbreaks to unsuccessful centroid via LoRA | Llama 3.1 8B · WildJailbreak held-out | jailbreak success rate: 46% -> 7% | Improved |
| SAE-informed LoRA jailbreak alignment | Llama 3.1 8B · MMLU | accuracy: 0.09% lower | No clear effect |
| SAE-informed LoRA jailbreak alignment | Llama 3.1 8B · OR-Bench | over-refusal rate: 3.5% -> 3.0% | No clear effect |
| align SAE features of ASR transcripts to clean counterparts via LoRA | Llama 3.1 8B · Spoken-SQuAD | exact match: 49.45 -> 58.33 | Improved |
| same SAE-aligned adapters applied to clean text | Llama 3.1 8B · SQuAD | exact match: 59.97 -> 67.88 | Improved |
| choose fine-tuning samples from middle energy deciles in middle layers (ablation) | GPT-2 25M · TinyStories typos | validation loss: U-shaped loss across deciles | Mixed |
Closest papers by meaning
- Understanding Transformers via N-Gram Statistics (NeurIPS 2024 · similarity 0.88)
- From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers (NeurIPS 2025 · similarity 0.88)
- Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study (ICLR 2025 · similarity 0.87)
- Transformers Learn Low Sensitivity Functions: Investigations and Implications (ICLR 2025 · similarity 0.87)
- Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits (NeurIPS 2025 · similarity 0.87)
- Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
