Research map / Architectures and efficiency / RNNs and normalization
Bilinear MLPs enable weight-based mechanistic interpretability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that bilinear MLPs (GLUs without nonlinearity) can be written as a third-order tensor, enabling weight-only eigendecomposition and SVD analyses that reveal low-rank, interpretable structure across MNIST, language models, and SAE feature circuits.
Problem
MLP element-wise nonlinearities make it hard to explain how MLP weights construct features and how computations trace through the layer, so MLPs are often treated as undecomposable in interpretability work.
Approach
Bilinear MLPs (g(x)=(Wx) �odot (Vx)) are expressible as a third-order bilinear tensor, so output-direction interaction matrices can be eigendecomposed exactly from the weights alone, giving input directions that explain each output feature, without relying on input data.
Key results
- Retaining only a handful of top eigenvectors per digit yields minimal accuracy drop (0.01%) on the MNIST model.
- Average correlation of SAE output feature activations with a single-eigenvector approximation starts around 0.65 and most features exceed 0.75 correlation with just two eigenvectors.
- Bilinear transformers match ReGLU/SwiGLU in constant-time loss (1.337 vs 1.337/1.336) but are 6% less data efficient than SwiGLU at constant epochs (1.337 vs 1.321).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SwiGLU with a bilinear MLP | 6-layer TinyStories transformer · TinyStories, constant epochs | loss: 1.321 -> 1.337 | Hurt |
| replace ReGLU with a bilinear MLP | 6-layer TinyStories transformer · TinyStories, constant epochs | loss: 1.332 -> 1.337 | Hurt |
| replace ReGLU with a bilinear MLP | 6-layer TinyStories transformer · TinyStories, constant time | loss: 1.337 -> 1.337 | No clear effect |
| replace SwiGLU with a bilinear MLP | 6-layer TinyStories transformer · TinyStories, constant time | loss: 1.336 -> 1.337 | No clear effect |
| train the SAE longer (ablation) | fw-medium layer 12 SAE, two-eigenvector feature approximation · FineWeb | correlation mean: 0.17 -> 0.59 | Helped |
| add dense Gaussian input noise (ablation) | bilinear MNIST image classifier · MNIST | feature interpretability | Helped |
| replace dense Gaussian noise with salt-and-pepper input noise (ablation) | bilinear image classifier · MNIST and Fashion-MNIST | accuracy | Hurt |
| use Perlin noise instead of dense Gaussian input noise (ablation) | bilinear image classifier · MNIST and Fashion-MNIST | feature interpretability | Hurt |
| add Gaussian noise to model activations (ablation) | shallow bilinear image classifier · image classification | experimental outcomes | No clear effect |
| add weight decay (ablation) | bilinear image classifier · image classification | eigenvalue sparsity | Helped |
| add dropout (ablation) | bilinear image classifier · image classification | overfitting | No clear effect |
| add translation augmentation (ablation) | bilinear image classifier · MNIST and Fashion-MNIST | overfitting | No clear effect |
Surprising findings
- Low-rank eigenvector approximations capture large SAE feature activations well despite the models being full-rank in general.
- Fine-tuning a pretrained SwiGLU model (TinyLlama) toward a bilinear form increases loss by only about 0.05 with continued training recovering most of the gap.
Limitations
- Eigenvectors are not guaranteed to be monosemantic, and orthogonality may limit interpretability for high-rank spectra.
- Methods rely on a set of meaningful output directions, which in deeper models come from input-dependent SAEs.
- Bilinear layers are inherently quadratic and need biases to model linear relations well.
Experimental setup
Datasets: MNIST Fashion-MNIST TinyStories FineWeb
Benchmarks: MNIST Fashion-MNIST TinyStories
Metrics: test accuracy cosine similarity of eigenvectors correlation of low-rank approximation with SAE feature activ cross-entropy loss misclassification rate
Baselines: SwiGLU ReGLU ReLU MLP
Builds on: Gated Linear Units sparse autoencoders TopK SAE SwiGLU bilinear layers
Closest papers by meaning
- Transcoders find interpretable LLM feature circuits (NeurIPS 2024 · similarity 0.85)
- The Weight Gram Matrix Captures Sequential Feature Linearization in Deep Networks (NeurIPS 2026 · similarity 0.84)
- Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $\mu$ Parametrization (ICML 2025 · similarity 0.83)
- Understanding MLP-Mixer as a wide and sparse MLP (ICML 2024 · similarity 0.83)
- Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders (NeurIPS 2025 · similarity 0.83)
- SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.