Research map / Architectures and efficiency / RNNs and normalization

Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Sparse dictionary representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows SAEs recover only ~9% of ground-truth features in a toy model, and that frozen-random SAE variants match fully-trained SAEs on interpretability, sparse probing, and causal editing for Gemma-2-2B and Llama-3-8B.

Problem

Whether sparse autoencoders actually recover meaningful, ground-truth features of neural network activations, given that standard proxy metrics (reconstruction, interpretability, sparse probing, causal editing) may not reflect true feature learning.

Approach

Two complementary sanity checks: a toy model with known ground-truth features at expansion factor 32, and three randomization baselines (Frozen Decoder, Soft-Frozen Decoder with cosine constraint tau=0.8, Frozen Encoder) that are evaluated against fully trained SAEs on real LLM activations.

Key results

• On the toy model with variable probabilities, BatchTopK SAE recovers only about 9% (297/3200) of ground-truth features at explained variance 0.71.
• At L0=160 on Gemma-2-2B layer 12, Soft-Frozen Decoder BatchTopK achieves AutoInterp 0.88 vs 0.90 for fully-trained SAE.
• Best frozen baseline causal editing (RAVEL) reaches about 0.73 vs 0.72 for fully-trained BatchTopK on Gemma-2-2B layer 12.

Surprising findings

• Soft-Frozen Decoder SAEs with random, nearly fixed decoder directions match fully-trained SAEs on interpretability, sparse probing, and causal editing.
• A TopK SAE recovers nearly all ground-truth features in the constant-probability toy setting, though this does not carry over to real activations.

Limitations

• Toy model assumes independent feature activations, omitting correlations present in real networks.
• Only standard SAE architectures (not transcoders or crosscoders) are evaluated with frozen baselines.
• Soft-Frozen Decoder tau=0.8 threshold was chosen heuristically; frozen baselines' gaps are larger for top-5 sparse probing.

Experimental setup

Datasets: OpenWebText synthetic toy model data (3200 ground-truth features, n=100) SAEBench sparse probing datasets RAVEL

Benchmarks: SAEBench sparse probing RAVEL AutoInterp

Metrics: explained variance feature recovery cosine similarity AutoInterp accuracy sparse probing accuracy RAVEL disentanglement score cross-entropy loss KL-divergence

Baselines: BatchTopK SAE JumpReLU SAE ReLU SAE TopK SAE Matryoshka SAE

Builds on: BatchTopK SAE JumpReLU SAE SAEBench RAVEL

Closest papers by meaning

• Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.92)
• Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control (ICLR 2025 · similarity 0.91)
• Sparse Autoencoders, Again? (ICML 2025 · similarity 0.91)
• Sparse Autoencoders Do Not Find Canonical Units of Analysis (ICLR 2025 · similarity 0.91)
• Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders (NeurIPS 2024 · similarity 0.91)
• Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders (ICML 2025 · similarity 0.91)