Research map / Architectures and efficiency / RNNs and normalization
Scaling and evaluating sparse autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Scales k-sparse (TopK) autoencoders on GPT-2 small and GPT-4 residual streams, reports scaling laws, few-dead-latent training fixes, and new feature-quality metrics, including a 16M-latent GPT-4 autoencoder.
Problem
Training sparse autoencoders at very large widths is hard because of the balance between reconstruction and sparsity objectives and many dead latents; there is also no good way to evaluate whether learned latents are useful features.
Approach
Use TopK activation to directly set the number of active latents, add an auxiliary dead-latent loss (AuxK) and transpose-tied encoder initialization to keep latents alive, fit joint scaling laws in latent count and sparsity, and evaluate latents with downstream loss, probe loss, N2G explanation precision/recall, and ablation sparsity.
Key results
- A 16 million latent TopK autoencoder trained on GPT-4 activations for 40B tokens, with only about 7% dead latents.
- Joint scaling law L(n,k) fit on GPT-4 autoencoders; MSE follows power laws in compute and number of latents with an irreducible loss term.
- TopK beats ReLU on the sparsity-MSE frontier and on probe loss and N2G F1, with better N2G recall but slightly worse precision.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TopK instead of ReLU activation | Sparse autoencoders · GPT-2 small | reconstruction-sparsity frontier | Helped |
| Increase the number of total latents | Sparse autoencoders · Feature-quality evaluation | feature quality | Helped |
| Increase L_0 | Sparse autoencoders · Feature explanation from token patterns | explanation quality | Hurt |
| Increase L_0 | Sparse autoencoders · Probe loss and ablation sparsity | probe loss and ablation sparsity | Helped |
| Use TopK rather than ReLU | Sparse autoencoders · N2G explanations | F1 score | Helped |
| Train with Multi-TopK instead of a single TopK(k) loss | TopK SAE · Progressive recovery across test-time k' | reconstruction MSE | Helped |
| Train with Multi-TopK instead of TopK at k | TopK SAE · Training at k | reconstruction | Hurt |
| Add tied encoder-decoder initialization (ablation) | TopK SAE · GPT-2 small, 128k latents, k=32 | MSE | Helped |
| Use the proposed encoder magnitude initialization (ablation) | TopK SAE with tied initialization · GPT-2 small | MSE | No clear effect |
| Use the proposed encoder magnitude initialization without tied initialization (ablation) | TopK SAE · GPT-2 small | MSE | Hurt |
| Remove the encoder bias b_enc (ablation) | TopK SAE · Convergence | convergence speed | Hurt |
| Add decoder column unit-norm normalization (ablation) | TopK SAE · GPT-2 small, 128k latents, k=32 | loss | Helped |
Surprising findings
- At k=512 on GPT-2 small, autoencoders learn many dense latents and scaling trends break down.
- Naive L1-trained ReLU autoencoders can be more efficient when paired with TopK or JumpReLU at test time, a result of progressive-code analysis.
Limitations
- TopK forces exactly k latents per token, which is likely suboptimal versus constraining expected L0.
- Precision evaluation relies on cheap N2G n-gram explanations rather than GPT-4-simulated explanations, and the probe metric is noisy and depends on assumed features.
- Context length of 64 tokens may be too short to show the most interesting GPT-4 behaviors.
Experimental setup
Datasets: GPT-2 small residual stream activations GPT-4 residual stream activations 61 binary classification datasets for probe evaluation
Benchmarks: downstream loss (KL divergence, delta cross-entropy) 1d probe loss on 61 binary classification tasks N2G explanation precision/recall/F1 ablation sparsity
Metrics: L0 sparsity normalized MSE downstream KL and delta cross-entropy probe cross-entropy N2G precision and recall effective number of vocab tokens affected
Baselines: ReLU sparse autoencoder with L1 penalty Gated SAE ProLU JumpReLU Neuron to Graph (N2G)
Builds on: k-sparse autoencoder sparse autoencoder ghost grads Gated SAE JumpReLU
Closest papers by meaning
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.92)
- Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders (NeurIPS 2024 · similarity 0.92)
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.91)
- Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.91)
- Semantic Optimal Transport for Sparse Autoencoder Feature Matching and Circuit Compression (NeurIPS 2026 · similarity 0.91)
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.