Research map / Large language models / Language, safety and interpretability
Measuring and Guiding Monosemanticity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the Feature Monosemanticity Score (FMS) to quantify monosemanticity of SAE latents, and G-SAE, a sparse autoencoder with a supervised conditioning loss that isolates labeled concepts in fixed latent dimensions, improving detection and steering of Llama3-8B.
Problem
Sparse autoencoder features are not reliably monosemantic and concepts are spread across many latents, with no standard metric for measuring monosemanticity and no guarantee that desired concepts are localized for detection and steering in LLMs.
Approach
A decision-tree-based metric (FMS) measures feature capacity, local and global disentanglement; G-SAE adds a binary cross-entropy conditioning loss that ties specific labeled concepts to fixed latent neurons during SAE training (or post-hoc finetuning), and steers by adding scaled decoder columns to the residual stream.
Key results
- G-SAE nearly doubles average FMS@1 from 0.27 (vanilla SAE) to 0.52.
- On toxicity steering, G-SAE achieves SuccessRate 0.98 vs 0.95 for vanilla SAE and 0.97 for ICV/DiffVec.
- G-SAE root-node accuracy on average 0.86 vs 0.70 for vanilla SAE; vanilla SAE needs about 41 features to match.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add latent conditioning on labeled concepts | Vanilla SAE training · Average across Toxicity, Shakespeare, and Privacy | FMS@1: 0.27 -> 0.52 | Helped |
| add latent conditioning on toxicity labels | SAE training · Toxicity | FMS@1: 0.26 -> 0.37 | Helped |
| add latent conditioning on Shakespeare-style labels | SAE training · Shakespeare | FMS@1: 0.28 -> 0.57 | Helped |
| add latent conditioning on privacy labels | SAE training · Privacy | FMS@1: 0.28 -> 0.62 | Helped |
| steer using conditioned SAE decoder features | Llama3-8B-base · Toxicity steering | SuccessRate: 0.95 -> 0.98 | Helped |
| steer using conditioned SAE decoder features | Llama3-8B-base · Shakespeare steering | SuccessRate: 0.64 -> 0.72 | Helped |
| steer using conditioned SAE decoder features | Llama3-8B-base · Privacy steering | SuccessRate: 0.47 -> 0.53 | Helped |
| finetune a pretrained SAE with G-SAE concept supervision | Pretrained SAE · Shakespeare | FMS@1: 0.37 -> 0.46 | Helped |
| finetune a pretrained SAE with G-SAE concept supervision | Pretrained SAE · Toxicity | FMS@1: 0.42 -> 0.49 | Helped |
| finetune a pretrained SAE with G-SAE concept supervision | Pretrained SAE · Privacy | FMS@1: 0.24 -> 0.33 | Helped |
| apply G-SAE during finetuning rather than pretraining (ablation) | G-SAE · Shakespeare | FMS@1: 0.37 -> 0.46 | Helped |
| apply G-SAE during finetuning rather than pretraining (ablation) | G-SAE · Toxicity | FMS@1: 0.42 -> 0.49 | Helped |
Surprising findings
- Vanilla SAE features show almost no accuracy drop when the top feature is removed, indicating concepts are spread across many latents.
- Applying G-SAE as post-hoc finetuning on a pretrained SAE improves monosemanticity while keeping SAE-Bench core metrics comparable.
Limitations
- Requires labeled concept data at training time, which may be scarce; supervising many concepts is impractical.
- Minor residual activations appear in semantically related neighboring concepts.
- Higher guidance weights improve interpretability but slightly degrade reconstruction.
Experimental setup
Datasets: RealToxicityPrompts Shakespeare pii-masking-300k RedPajama v2
Benchmarks: SAE-Bench RealToxicityPrompts Shakespeare pii-masking-300k
Metrics: FMS FMS_local FMS_global accs_0 SuccessRate win rate CE loss score MSE Perspective API score Presidio privacy violations reduction
Baselines: Vanilla SAE Model Arithmetic PreADD ICV DiffVec Llama3-8B-base
Builds on: sparse autoencoders TopK SAE Mutual Information Gap PreADD In-context vectors
Closest papers by meaning
- Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words (ICLR 2025 · similarity 0.91)
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.90)
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.89)
- Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework (NeurIPS 2025 · similarity 0.88)
- Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness (ICLR 2025 · similarity 0.88)
- SASA: Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.