Research map / Architectures and efficiency / RNNs and normalization
Learning a Unified Cross-Model Semantic Dictionary via Gated Bottleneck Sparse Autoencoders
Jinchi Zhu, Tie Luo
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes learning a unified cross-model semantic dictionary using gated bottleneck sparse autoencoders that separate shared semantic concepts from model-specific biases.
Problem
Representations of foundation models pretrained on similar data encode aligned concepts, but are entangled with model-specific inductive biases, hindering unified interpretability and cross-model feature reuse.
Approach
Uses gated bottleneck sparse autoencoders to learn a shared dictionary across models that explicitly accounts for model-specific biases, unlike prior methods that align concepts across model-specific spaces without modeling those biases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GB-SAE shared dictionary reconstructions instead of USAE/SPARC reconstructions or projected features | linear ImageNet-1K classifier · ImageNet-1K (Config I) | Acc@1/Acc@5 mean: 0.78/0.93 (SPARC) -> 0.86/0.97 | Improved |
| use GB-SAE shared dictionary instead of pairwise cross-reconstruction of baselines | cross-model reconstruction · ImageNet-1K (Config I) | R2 cross-model: R2-cross 0.20 (USAE), 0.41 (SPARC) -> R2-sh 0.53 | Improved |
| add private branch to shared branch for reconstruction (ablation) | GB-SAE · ImageNet-1K (Config I) | R2: 0.53 -> 0.60 | Improved |
| retrieve in shared reconstructed space instead of private or projected features (ablation) | zero-shot cross-model retrieval · ImageNet-1K val | recall: up to two orders of magnitude | Improved |
| reuse frozen shared decoder with a learned linear adapter for unseen CLIP | GB-SAE · ImageNet-1K, CLIP | R2-shared: R2-sh 0.48/0.50; R2-total 0.83 | Improved |
| increase lambda_align relative to lambda_sh-only (ablation) | GB-SAE · ImageNet-1K | reconstruction quality | Hurt |
| increase lambda_sh-only (ablation) | GB-SAE · ImageNet-1K | cross-model retrieval | Hurt |
| increase lambda_gate too far (ablation) | GB-SAE · ImageNet-1K | alignment | Hurt |
| add ConvNeXt as fourth backbone (Config II to III shared R2) (ablation) | GB-SAE · ImageNet-1K | mean R2-sh: 0.58 -> 0.54 | Hurt |
Experimental setup
Builds on: sparse autoencoder
Closest papers by meaning
- Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures (NeurIPS 2026 · similarity 0.91)
- Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment (ICML 2025 · similarity 0.91)
- Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models (ICML 2025 · similarity 0.90)
- Semantic Optimal Transport for Sparse Autoencoder Feature Matching and Circuit Compression (NeurIPS 2026 · similarity 0.90)
- Learning Multi-Level Features with Matryoshka Sparse Autoencoders (ICML 2025 · similarity 0.89)
- Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
