AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Learning a Unified Cross-Model Semantic Dictionary via Gated Bottleneck Sparse Autoencoders

Jinchi Zhu, Tie Luo

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Layer-wise analysis of language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes learning a unified cross-model semantic dictionary using gated bottleneck sparse autoencoders that separate shared semantic concepts from model-specific biases.

Problem

Representations of foundation models pretrained on similar data encode aligned concepts, but are entangled with model-specific inductive biases, hindering unified interpretability and cross-model feature reuse.

Approach

Uses gated bottleneck sparse autoencoders to learn a shared dictionary across models that explicitly accounts for model-specific biases, unlike prior methods that align concepts across model-specific spaces without modeling those biases.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use GB-SAE shared dictionary reconstructions instead of USAE/SPARC reconstructions or projected featureslinear ImageNet-1K classifier · ImageNet-1K (Config I)Acc@1/Acc@5 mean: 0.78/0.93 (SPARC) -> 0.86/0.97Improved
use GB-SAE shared dictionary instead of pairwise cross-reconstruction of baselinescross-model reconstruction · ImageNet-1K (Config I)R2 cross-model: R2-cross 0.20 (USAE), 0.41 (SPARC) -> R2-sh 0.53Improved
add private branch to shared branch for reconstruction (ablation)GB-SAE · ImageNet-1K (Config I)R2: 0.53 -> 0.60Improved
retrieve in shared reconstructed space instead of private or projected features (ablation)zero-shot cross-model retrieval · ImageNet-1K valrecall: up to two orders of magnitudeImproved
reuse frozen shared decoder with a learned linear adapter for unseen CLIPGB-SAE · ImageNet-1K, CLIPR2-shared: R2-sh 0.48/0.50; R2-total 0.83Improved
increase lambda_align relative to lambda_sh-only (ablation)GB-SAE · ImageNet-1Kreconstruction qualityHurt
increase lambda_sh-only (ablation)GB-SAE · ImageNet-1Kcross-model retrievalHurt
increase lambda_gate too far (ablation)GB-SAE · ImageNet-1KalignmentHurt
add ConvNeXt as fourth backbone (Config II to III shared R2) (ablation)GB-SAE · ImageNet-1Kmean R2-sh: 0.58 -> 0.54Hurt

Experimental setup

Builds on: sparse autoencoder

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy