Research map / Large language models / Language, safety and interpretability

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability

ICML 2025 · Large language models › Language, safety and interpretability · approach: Linguistic encoding in LLM representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SAEBench, an eight-metric benchmark suite for sparse autoencoders covering concept detection, interpretability, reconstruction and feature disentanglement, and releases over 200 SAEs trained across seven architectures on Gemma-2-2B and Pythia-160M.

Problem

Sparse autoencoders are typically evaluated with unsupervised proxy metrics (sparsity-fidelity tradeoff) whose relevance to practical interpretability is unclear, so progress is hard to measure.

Approach

A standardized multi-metric evaluation suite (sparse probing, feature absorption, RAVEL, unlearning, SCR, TPP, automated interpretability, loss recovered) plus two new zero-ablation disentanglement metrics (SCR and TPP), applied to a large sweep of SAE architectures, widths and sparsities; shows that proxy metric gains do not reliably transfer to downstream metrics.

Key results

• Matryoshka BatchTopK SAEs achieve best scores on 5 of 8 metrics in typical L0 range 40-200, despite worse sparsity-fidelity frontier.
• Matryoshka is the only architecture whose feature disentanglement (absorption, SCR) improves with dictionary width; other architectures show inverse scaling.
• The ReLU SAE is outperformed by other methods on 5 of 8 metrics and performs worst on loss recovered.

Measured effects reported in the paper

Surprising findings

• Sparse probing provides limited differentiation between architectures, widths and sparsities, with scores in a narrow range.
• Feature disentanglement metrics (TPP, RAVEL) prefer higher L0 values (>400), diverging from the conventional 20-200 range.

Limitations

• Supervised metrics only cover concepts with ground truth data, so they examine a small fraction of latents and some are noisy.
• Unlearning evaluation is constrained because Gemma-2-2B has weak baseline performance on most unlearning test sets.
• Results are limited to Gemma-2-2B and Pythia-160M and a single middle layer per model.

Experimental setup

Datasets: The Pile OpenWebText WMDP-bio WikiText MMLU bias_in_bios Amazon Reviews Europarl GitHub AG News RAVEL

Benchmarks: SAEBench sparse probing feature absorption RAVEL unlearning (WMDP-bio) spurious correlation removal targeted probe perturbation automated interpretability loss recovered

Metrics: loss recovered L0 sparsity accuracy interpretability score absorption score disentanglement score

Baselines: ReLU SAE TopK SAE BatchTopK SAE Gated SAE JumpReLU SAE P-Annealing SAE MatryoshkaBatchTopK SAE PCA

Builds on: RAVEL SHIFT Feature Absorption Gated SAE TopK SAE Matryoshka SAE JumpReLU SAE Gemma Scope

Closest papers by meaning

• Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders (NeurIPS 2024 · similarity 0.94)
• Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.93)
• Scaling and evaluating sparse autoencoders (ICLR 2025 · similarity 0.92)
• Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words (ICLR 2025 · similarity 0.92)
• Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.92)
• Low-Rank Adapting Models for Sparse Autoencoders (ICML 2025 · similarity 0.92)