Research map / Large language models / Language, safety and interpretability
Evaluating SAE interpretability without generating explanations
Gonçalo Paulo, Nora Belrose
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| finetune the embedding model on highlighted-token contexts (ablation) | example embedding scoring · Correlation with human intruder scores (112 latents) | Pearson correlation: 0.42 -> 0.78 | Improved |
| sample activating examples from the highest activation decile instead of the lowest | intruder detection · SmolLM2 SAE latents | accuracy: up to 20% higher accuracy | Improved |
| use a larger evaluator LLM (Llama 3.1 70b instead of 8b) (ablation) | intruder detection · Correlation with human intruder scores (56 latents) | Pearson correlation: 0.52 -> 0.76 | Improved |
| use Claude Sonnet 3.5 instead of Llama 3.1 8b as intruder judge (ablation) | intruder detection · Correlation with human intruder scores (56 latents) | Pearson correlation: 0.52 -> 0.83 | Improved |
| use intruder detection instead of detection scoring | LLM scoring · Correlation with human intruder scores (105 latents) | Pearson correlation: 0.48 -> 0.75 | Improved |
| use example embedding scoring instead of LLM intruder detection | scoring method · Correlation with human intruder scores (105 latents) | Pearson correlation: 0.75 -> 0.78 | No clear effect |
| show an intruder from a different decile instead of non-activating examples | intruder detection · Deciles intruder task | accuracy: near random for close deciles | Hurt |
| apply to other SAE families (Gemmascope, Pythia transcoders) | example embedding scoring · Other sparse coders | Pearson correlation: 0.84 (Gemma 2 9b), 0.67 (Pythia-160m) with LLM intruder | No clear effect |
Closest papers by meaning
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.92)
- Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.92)
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.91)
- Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words (ICLR 2025 · similarity 0.91)
- Transformer Key-Value Memories Are Nearly as Interpretable as Sparse Autoencoders (NeurIPS 2025 · similarity 0.91)
- Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
