Research map / Architectures and efficiency / RNNs and normalization
Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes evaluating sparse autoencoders against supervised, attribute-based feature dictionaries on the GPT-2 Small indirect object identification task, using sufficiency/necessity, sparse controllability, and interpretability tests.
Problem
Sparse autoencoder features lack ground truth in realistic LLM settings, so existing SAE evaluations are indirect and it is unclear whether learned features are useful for explaining or controlling model computation.
Approach
Build supervised feature dictionaries from prompt attributes (IO, S, Pos) by averaging activations per attribute value, use them as skylines, and evaluate unsupervised SAE dictionaries along three axes: reconstruction sufficiency/necessity, sparse attribute controllability, and interpretability against causal roles.
Key results
- Supervised mean feature dictionaries achieve >80% agreement with ground-truth interventions for attribute edits across circuit cross-sections, using a single feature swap.
- Task-specific SAEs edit Pos and, to a small extent, IO, but need more features and larger edit magnitudes than supervised dictionaries; full-distribution SAEs typically need 32+ features.
- SAE features show feature occlusion (S features hidden by higher-magnitude IO features in L10H0 queries) and feature over-splitting (Pos split into >=10 features).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use supervised mean feature dictionaries instead of task-specific SAEs | IOI feature dictionaries · IOI | sufficiency and necessity of reconstructions | Helped |
| Use supervised mean feature dictionaries instead of full-distribution SAEs | IOI feature dictionaries · IOI | sufficiency and necessity of reconstructions | Helped |
| Train SAEs on IOI data rather than OpenWebText | GPT-2 Small SAE features · IOI | sparse controllability | Helped |
| Use supervised feature dictionaries instead of task-specific SAEs for attribute editing | IOI circuit cross-sections · IOI | attribute-edit agreement with ground-truth intervention | Helped |
| Use task-specific SAEs rather than full-distribution SAEs | IOI feature interpretability · IOI | interpretability of features | No clear effect |
| Use high-F1 interpretable features to guide task-specific SAE edits | Task-specific SAEs · IOI | sparse controllability | No clear effect |
| Freeze decoder directions at initialization (ablation) | Task-specific SAE · IOI | sparse controllability | Hurt |
| Reduce the magnitude of supervised IO features in activations | L10H0 name mover queries · IOI | discovery of interpretable S-name features | Helped |
| Use task-specific SAEs to edit Pos rather than S | Task-specific SAE features · IOI | sparse controllability | Mixed |
Surprising findings
- Task-specific SAEs with frozen random decoder directions perform on par with full-distribution SAEs on controllability.
- Pos binary attribute is over-split into 10 or more mostly non-overlapping features by SAEs.
Limitations
- Relies on a subjectively chosen set of attributes for supervision.
- Only one task (IOI) and one language model (GPT-2 Small) are studied.
- SAE hyperparameter tuning was not exhaustive.
Experimental setup
Datasets: IOI dataset (216 names, 144 objects, 75 places, templated sentences) OpenWebText
Benchmarks: IOI task logit difference
Metrics: necessity score sufficiency (normalized logit difference) edit accuracy agreement with ground-truth intervention F1 score for interpretability
Baselines: mean ablation full-distribution SAEs frozen-decoder task SAEs
Builds on: sparse autoencoders IOI circuit analysis linear representation hypothesis faithfulness/completeness evaluation
Closest papers by meaning
- Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry (NeurIPS 2025 · similarity 0.91)
- Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines? (NeurIPS 2026 · similarity 0.91)
- Learning Multi-Level Features with Matryoshka Sparse Autoencoders (ICML 2025 · similarity 0.90)
- Efficient Dictionary Learning with Switch Sparse Autoencoders (ICLR 2025 · similarity 0.90)
- Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment (ICML 2025 · similarity 0.89)
- From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.