AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

What Does a Sparse Autoencoder Feature Do? A Weight-Based Account

Yiting Liu, Zhihong Deng

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Sparse dictionary representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a weight-based account of sparse autoencoder features that characterizes the computational roles features inherit from the SAE training objective, rather than inferring semantics only from activation patterns.

Problem

Current SAE interpretation pipelines infer feature semantics from activation patterns, implicitly assuming every feature has a semantic explanation and overlooking computational roles inherited from the SAE training objective.

Approach

Interprets SAE features through their encoder/decoder weights rather than activation patterns, arguing some features may serve computational roles set by the training objective instead of carrying semantic meaning.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Zero out a weight-identified semantic feature (do(z=0)) vs matched-norm non-semantic feature (ablation)Gemma-2-9B and Llama-3.1-8B SAE features · next-token logit at top-1 tokenmedian |delta logit|: 0.8-7.1 vs 0.02-0.13Improved
Zero out weight-identified query specialists / key hubs vs matched baselines (ablation)SAE features · next-layer attentioneffect on target attention quantity: 43x-2161x (queries), 9x-37x (keys)Improved
Use untied embeddings (Llama) instead of tied (Gemma)SAE semantic depth profile · Llama-3.1-8B vs Gemma-2pass rate across depthMixed
Ablate any single of the three semantic metrics (ablation)three-metric classifier · Gemma-2-2B/9Bjoint pass rateMixed
Vary calibration percentile p in [40,60] (ablation)classifier · six model-projection combinationsSpearman with p=50 profile: minimum 0.878No clear effect
Apply per-feature normalization vs unnormalized baseline (ablation)attention-participation scores · Gemma-2-9BSpearman rank correlation: 0.902-0.985 (queries), 0.876-0.980 (keys)No clear effect
Compare semantic vs non-semantic features on attention weightSAE features · Gemma-2 query-side attentionattention weightHurt
Compare semantic vs non-semantic features in Llama input phaseSAE features · Llama-3.1-8B layers before 11attention weightImproved
Use Gemma-2 instead of Llama for attention specialist countsSAE features · specialist populations by depth2-layer oscillationMixed

Experimental setup

Builds on: sparse autoencoders

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy