Research map / Architectures and efficiency / RNNs and normalization
Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PS-Eval, a polysemy-based evaluation suite (WiC-derived dataset plus confusion-matrix metrics) that tests whether sparse autoencoder features separate word meanings in context, and finds that MSE-L0-optimal SAE variants do not necessarily improve this semantic metric.
Problem
Standard SAE evaluation relies on MSE and L0 sparsity (and IOI or GPT-4 automated checks), which do not directly measure whether SAE features extract monosemantic features from polysemantic LLM activations for polysemous words.
Approach
Build PS-Eval from WiC word pairs with same/different meanings in context; feed a templated prompt to GPT-2 small, take the max-activating SAE feature for the target token in each context, and score whether same-meaning pairs share a feature (recall/precision) and different-meaning pairs do not (specificity), summarized in a confusion matrix.
Key results
- Accuracy on PS-Eval rises with SAE expand ratio, saturating around expand ratio 64.
- F1 ranking across activation functions is ReLU > JumpReLU > TopK, so MSE-L0-optimized variants do not improve polysemy separation.
- Specificity improves in deeper layers (from about layer 6) even as MSE and L0 rise, and attention-output SAEs show markedly higher specificity than residual or MLP SAEs despite lower F1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase the SAE expansion ratio (ablation) | GPT-2 small SAE · PS-Eval | accuracy | Helped |
| increase the SAE expansion ratio beyond around 64 (ablation) | GPT-2 small SAE · PS-Eval | accuracy | No clear effect |
| replace ReLU with JumpReLU (ablation) | GPT-2 small SAE · PS-Eval | F1 score | Hurt |
| replace ReLU with TopK (ablation) | GPT-2 small SAE · PS-Eval | F1 score | Hurt |
| use smaller TopK k (ablation) | GPT-2 small SAE · PS-Eval | recall | Helped |
| use JumpReLU with STE (ablation) | GPT-2 small SAE · PS-Eval | F1 score | No clear effect |
| train the SAE on deeper layers (ablation) | GPT-2 small SAE · PS-Eval | specificity | Helped |
| train the SAE on Attention output instead of Residual or MLP output (ablation) | GPT-2 small SAE · PS-Eval | specificity | Helped |
| train the SAE on Attention output instead of Residual or MLP output (ablation) | GPT-2 small SAE · PS-Eval | F1 score | Hurt |
| remove Ghost Grad (ablation) | GPT-2 small SAE · PS-Eval | accuracy | Hurt |
| add Ghost Grad (ablation) | GPT-2 small SAE · dead latent count | number of dead latents | Helped |
| remove the L1 sparsity constraint (ablation) | GPT-2 small SAE · PS-Eval | recall | Hurt |
Surprising findings
- Activation functions designed to improve the MSE-L0 Pareto frontier (TopK, JumpReLU) performed worse than plain ReLU on the polysemy metrics.
- Specificity is inflated by construction: a random SAE reaches specificity near 1.0 while recall is near 0, so specificity must be read alongside other metrics.
Limitations
- Evaluation relies on the single maximum-activating feature, which may not reflect the intended word meaning for all samples or models.
- Experiments are mostly on GPT-2 small layer 4, and open SAEs for other models perform worse, with parameters possibly needing per-model tuning.
- Specificity can be trivially high for randomly activating SAEs, so the metric needs holistic interpretation.
Experimental setup
Datasets: Word-in-Context (WiC) RedPajama
Benchmarks: PS-Eval
Metrics: accuracy precision recall F1 specificity MSE L0
Baselines: ReLU SAE TopK SAE JumpReLU SAE Dense SAE Random SAE
Builds on: sparse autoencoders TopK SAE JumpReLU Word-in-Context logit lens
Closest papers by meaning
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.92)
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.92)
- Measuring and Guiding Monosemanticity (NeurIPS 2025 · similarity 0.91)
- Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders (NeurIPS 2024 · similarity 0.90)
- Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.