Research map / Large language models / Language, safety and interpretability
A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies "feature absorption" in sparse autoencoders, where hierarchical features are absorbed into child latents so parent latents fail to fire on some tokens, and measures it across Gemma Scope and self-trained SAEs.
Problem
SAEs are assumed to recover true interpretable features, but when features form hierarchies, seemingly monosemantic latents fail to fire where they should, undermining their reliability as classifiers.
Approach
Sparsity pressure in SAEs makes a parent latent absorb the direction of a co-occurring child feature, so the child-specific latent fires instead and the parent latent has gerrymandered false negatives; toy models and a proof show absorption lowers L1 loss for hierarchical features.
Key results
- Gemma Scope SAEs reach no F1 match to the logistic regression probe on first-letter classification; the best latent (layer 3, 16k, L0=59) has F1 0.81 but misses tokens like ' short'.
- Feature absorption rate increases with higher sparsity (lower L0) and larger SAE width in Gemma Scope 16k and 65k SAEs.
- Absorption also appears in self-trained L1 and TopK SAEs on Llama 3.2 1B and Qwen2 0.5B, and in toy models including a batch TopK SAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use hierarchical feature co-occurrence instead of independent feature firing | SAE toy model · Toy model with four features | feature absorption | Helped |
| Increase feature co-occurrence between hierarchical features | SAE · Hierarchical feature toy model | sparsity loss: dL_sp^(1,2)/dδ = -p_11 | Helped |
| Increase SAE sparsity | Gemma Scope SAEs · First-letter task | feature absorption rate | Hurt |
| Increase SAE width | Gemma Scope SAEs · First-letter task | feature absorption rate | Hurt |
| Use a larger SAE L0 | Gemma Scope SAEs · First-letter classification | precision and recall | Mixed |
| Use a single SAE latent instead of the LR probe | First-letter classification · Gemma Scope SAEs | F1 | Hurt |
| Add a second latent for the uppercase/lowercase split | K-sparse probe on SAE activations · Letter L classification | F1 | Helped |
| Add variance to the parent feature's firing magnitude | SAE toy model · Hierarchical features with parent magnitude variance 0.1 | feature absorption | No clear effect |
| Allow the child feature to fire without the parent in 5% of cases | SAE toy model · Imperfect hierarchical co-occurrence | feature absorption | No clear effect |
| Use a batch TopK SAE instead of an L1 SAE | SAE toy model · Hierarchical features | feature absorption | No clear effect |
| Remove the probe direction from latent 1085 before ablating it (ablation) | Gemma Scope layer 3, width 16k, L0 59 SAE · Token '_short' on the first-letter task | ablation effect | Hurt |
Surprising findings
- Varying SAE size or sparsity is insufficient to eliminate absorption, since higher L0 SAEs have lower absorption but still show splitting and lower precision.
- Absorption occurs even with TopK SAEs that have no L1 loss term, as absorption increases sparsity and improves reconstruction at fixed k.
Limitations
- Absorption metric relies on ablation, so it cannot detect absorption past layer 17 in Gemma-2-2B.
- Metric is conservative and likely underestimates absorption, since it ignores multiple absorbing latents or weakly firing main latents.
- Only a small number of non-JumpReLU SAEs were trained due to compute constraints.
Experimental setup
Datasets: synthetic toy feature activations residual stream activations on first-letter ICL prompts
Benchmarks: first-letter identification task
Metrics: F1 precision recall feature splitting count feature absorption rate
Baselines: logistic regression linear probe k=1 sparse probing
Builds on: sparse autoencoders k-sparse probing JumpReLU Gemma Scope SAELens
Closest papers by meaning
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.91)
- Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders (NeurIPS 2024 · similarity 0.91)
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.91)
- Scaling and evaluating sparse autoencoders (ICLR 2025 · similarity 0.91)
- SASA: Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability (NeurIPS 2026 · similarity 0.90)
- Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.