Research map / Architectures and efficiency / RNNs and normalization

Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders

NeurIPS 2025 · Architectures and efficiency › RNNs and normalization · approach: Layer-wise analysis of language models · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Mixture of Decoders (MxD), a Hadamard-factorized layer-level sparse MoE that replaces dense MLP/GLU layers in LLMs with tens of thousands of full-rank linear experts, achieving better reconstruction than Transcoders at matched sparsity.

Problem

Sparse MLP approximations (Transcoders, Skip Transcoders) that learn interpretable features via neuron-level sparsity substantially hurt model next-token loss when replacing dense MLPs, limiting faithfulness and practical use as layer replacements.

Approach

Replace neuron-level sparsity with layer-level sparsity: a sparsely gated mixture of linear decoder experts whose weight tensor is Hadamard-factorized (W_n = D diag(c_n)), so each expert is full-rank (rank of the original decoder) and the forward pass reduces to (C^T a) * (D^T z), keeping parameters O(N+H) instead of NHO.

Key results

• MxDs achieve lower downstream cross-entropy loss than Transcoders and Skip Transcoders at all sparsity levels K across 4 LLMs (GPT2-124M, Pythia-410M, Pythia-1.4B, Llama-3.2-3B) with up to 3B parameters.
• At small K, MxD normalized MSE is up to an order of magnitude smaller than Transcoders' normalized MSE.
• Learned MxD expert matrices have mean normalized rank 0.99 across the four models, i.e. nearly full rank.

Measured effects reported in the paper

Surprising findings

• Learned MxDs naturally develop a shared expert performing a common base transformation, with the remaining K-1 active experts specializing.
• Sparse MoEs with equivalent expert counts would need about 2.16 trillion parameters for a single layer on Llama-3.2-3B, orders of magnitude above the base model.

Limitations

• Experiments only directly cover LLMs up to 3B parameters (plus initial Gemma2-27B results), limited by compute.
• Gating and large encoders add inference-time cost; TopK reduces decoder FLOPs but not encoder/gate cost.
• Steering evaluation relies on LLM judges whose reliability is uncertain; no error bars due to compute limits.

Experimental setup

Datasets: OpenWebText SAEBench probing tasks

Benchmarks: SAEBench sparse probing (24 tasks) news category probing feature steering

Metrics: downstream cross-entropy loss normalized MSE F1 score textual coherence steerability future-token identity rate normalized expert rank

Baselines: Transcoders Skip Transcoders TopK SAEs muMoE (CP low-rank MoE)

Builds on: Transcoders Skip Transcoders TopK SAE GLU muMoE

Closest papers by meaning

• The Sparsity Whisperer (NeurIPS 2026 · similarity 0.89)
• Adaptive Layer Sparsity for Large Language Models via Activation Correlation Assessment (NeurIPS 2024 · similarity 0.88)
• Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective (ICML 2025 · similarity 0.88)
• Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.88)
• Layer by Layer: Uncovering Hidden Representations in Language Models (ICML 2025 · similarity 0.88)
• Mixture of Experts Made Intrinsically Interpretable (ICML 2025 · similarity 0.87)