Research map / Architectures and efficiency / RNNs and normalization
Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Gated Sparse Autoencoders, which decouple feature detection (gated encoder trained with L1) from magnitude estimation, yielding a Pareto improvement in sparsity vs. reconstruction fidelity and resolving shrinkage on LM activations.
Problem
Sparse autoencoders trained with an L1 penalty for interpretable feature discovery in language model activations suffer from shrinkage and a biased trade-off between reconstruction fidelity and sparsity.
Approach
Replace the single-layer ReLU SAE encoder with a gated encoder whose Heaviside-gated path detects active features and is the only path penalized by L1, while a separate magnitude path estimates activation sizes; weights are tied between paths and an auxiliary reconstruction loss with frozen decoder trains the gate.
Key results
- Gated SAEs are a Pareto improvement over baseline SAEs in loss recovered vs L0 at all sites tested across GELU-1L, Pythia-2.8B and Gemma-7B.
- Gated SAEs resolve shrinkage: relative reconstruction bias gamma is about 1, while baseline SAEs show gamma < 1 that worsens as L0 decreases.
- In a blinded human study, Gated SAE features are similarly interpretable to baseline features, with mean difference 0.13 (90% CI [0, 0.26]) favoring Gated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the baseline SAE with a Gated SAE | SAE reconstruction of language-model activations · GELU-1L, Pythia-2.8B, and Gemma-7B tested sites | loss recovered at fixed L0 | Helped |
| replace the baseline SAE with a Gated SAE | SAE reconstruction of language-model activations · GELU-1L and Pythia-2.8B | relative reconstruction bias γ | Helped |
| replace the baseline SAE with a Gated SAE | SAE features · Pythia-2.8B and Gemma-7B | interpretability label score: 0.13 (90% CI [0, 0.26]); p = 0.06 | No clear effect |
| unfreeze decoder weights in the auxiliary loss Laux (ablation) | Gated SAE · GELU-1L MLP neuron activations | loss recovered | Hurt |
| remove the rmag scaling parameter (ablation) | Gated SAE · GELU-1L MLP neuron activations | loss recovered | Hurt |
| untie gating and magnitude encoder parameters (ablation) | Gated SAE · GELU-1L MLP neuron activations | loss recovered | Hurt |
| learn rescale and shift while freezing baseline encoder and decoder directions | Baseline SAE · GELU-1L MLP neuron activations | loss recovered | Helped |
| use a Gated SAE rather than baseline + rescale & shift | SAE reconstruction · GELU-1L MLP neuron activations | loss recovered | Helped |
Surprising findings
- Resolving shrinkage alone (baseline + rescale & shift) explains only a small part of the performance gain, suggesting Gated SAEs also learn better encoder and decoder directions.
- Baseline SAE dictionaries evaluated with inference-time optimization often perform best at a test-time L0 far above their training L0.
Limitations
- Benchmarking focused on GELU-1L and Pythia/Gemma model families, so generalization to other families is uncertain.
- Human interpretability does not necessarily imply usefulness for downstream mechanistic interpretability tasks such as circuit analysis or steering.
- Small double-blind interpretability study with only 5 to 7 raters and marginal significance for interpretability gain.
Experimental setup
Datasets: LM activations from GELU-1L, Pythia-2.8B and Gemma-7B forward passes
Benchmarks: loss recovered vs L0 on GELU-1L, Pythia-2.8B, Gemma-7B activations
Metrics: L0 loss recovered relative reconstruction bias (gamma) interpretability human rating
Baselines: baseline SAE (ReLU + L1) Wright and Sharkey rescale & shift Taggart JumpReLU (parallel work)
Builds on: sparse autoencoders (Bricken et al. 2023) Gated Linear Units JumpReLU
Closest papers by meaning
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.94)
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.93)
- Low-Rank Adapting Models for Sparse Autoencoders (ICML 2025 · similarity 0.93)
- Scaling and evaluating sparse autoencoders (ICLR 2025 · similarity 0.92)
- Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations (ICML 2025 · similarity 0.92)
- Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders (ICML 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.