Research map / Architectures and efficiency / RNNs and normalization
User Embeddings are Superpositions of Interpretable Behavioral Modes
Jonathan Kozaczuk, Zheng Dong, Noyan Evirgen, Keivan Majidi, Eddie K Ng, Chris Porter, Qi Zhao, Jia Geng, Changmao Li, Brandon Feng
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes that user embeddings are sparse superpositions of interpretable behavioral modes, making dense user representations interpretable.
Problem
User embeddings used in recommendation and personalization are dense and opaque, limiting interpretability of the behavioral information they encode.
Approach
Treats user embeddings as sparse superpositions of interpretable behavioral modes, a structure the authors state is directly predicted by the superposition hypothesis from interpretability research.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Merge SAE features with decoder cosine similarity above 0.45 (ablation) | SAE · Synthetic mean-pooled embeddings (m=128) | median per-feature Pearson r: about 0.35 -> about 0.70 | Improved |
| Replace K-Means encoding with SAE | Behavioral-mode decomposition · Open e-commerce category tasks | Lift: 1.85x -> 1.93x | Improved |
| Replace K-Means encoding with SAE | Behavioral-mode decomposition · MovieLens (matched size and L0) | Lift and AUROC | No clear effect |
| Replace K-Means encoding with SAE | Behavioral-mode decomposition · MovieLens SASRec | coverage: 0.811 -> 0.977 | Improved |
| Use SAE instead of a text-embedding baseline | NL audience retrieval · Open e-commerce life-event tasks | Lift: 2.13x -> 5.30x | Improved |
| Use SAE instead of PCA | Behavioral-mode recovery · Synthetic SASRec embeddings | modes recovered: 0.00 -> 0.97 | Improved |
| Add capacity to K-Means (512 components instead of 75) (ablation) | K-Means encoding · Synthetic mean-pooled embeddings | code-GT alignment | No clear effect |
| Freeze the decoder (soft-frozen decoder SAE) instead of training it (ablation) | SAE · Synthetic data | feature recovery | Mixed |
| Use FastICA instead of SAE | Mode decomposition · Synthetic data | per-feature monosemanticity: 0.53 -> 0.75 | Improved |
| Replace LLM used for interpretation with open-weight Llama 3.3 70B (ablation) | Auto-interpretation · Open e-commerce (one seed) | SAE retrieval | No clear effect |
Experimental setup
Builds on: superposition hypothesis
Closest papers by meaning
- Long-Sequence Recommendation Models Need Decoupled Embeddings (ICLR 2025 · similarity 0.87)
- ImplicitSLIM and How it Improves Embedding-based Collaborative Filtering (ICLR 2024 · similarity 0.86)
- On the Embedding Collapse when Scaling up Recommendation Models (ICML 2024 · similarity 0.86)
- FACE: A General Framework for Mapping Collaborative Filtering Embeddings into LLM Tokens (NeurIPS 2025 · similarity 0.85)
- UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence (NeurIPS 2026 · similarity 0.85)
- Language Representations Can be What Recommenders Need: Findings and Potentials (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
