Research map / Architectures and efficiency / RNNs and normalization
SVD as a Fast Interpretability Method for Transformers
Min Xue, Artur Andrzejak
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reconstruct W_down with top-20 SCA subspaces only (sufficiency) | NaNA · Dgene (GPT-2 family, middle-to-late layers) | top-1 accuracy: 97-99% | Improved |
| ablate top-k high-contribution subspaces (ablation) | NaNA · Dgene (GPT-2 family) | top-1 accuracy, probability, rank | Hurt |
| ablate random DEU subspaces instead of top-k (ablation) | NaNA · GPT2-Medium positive semantics case | target rank | No clear effect |
| remove top 10 subspaces (ablation) | NaNA · GPT2-Medium 'The cat looks very' | top predictions | Hurt |
| reconstruct with only top-5 subspaces (ablation) | NaNA · GPT2-Medium 'The cat looks very' | target token recovery | Improved |
| amplify subspace 7 of layer 16 (alpha from 0 to 8) (ablation) | NaNA · GPT2-Medium 'The cat looks very' | top tokens | Improved |
| suppress subspace 7 (alpha below 0) (ablation) | NaNA · GPT2-Medium 'The cat looks very' | top tokens | Hurt |
| use SCA subspaces instead of SAE and TransCoder features beyond k>60 | NaNA · GPT-2 layer 10 | target rank | Improved |
| NaNA instead of SAE/TransCoder for single matrix runtime | NaNA · GPT-2 single projection matrix | runtime: ≈30 min -> < 2 min | Improved |
| NaNA instead of SAE/TransCoder GPU memory | NaNA · GPT-2 single projection matrix | GPU memory: ≈16 GB -> < 1 GB | Improved |
| NaNA instead of precomputing SAE/TransCoder modules for GPT2-XL | NaNA · GPT2-XL pathway extraction | runtime: over 24 hours -> 8.59 minutes | Improved |
| add more subspaces to reconstruction (around k=30) (ablation) | NaNA · GPT-2 layer 10 | target probability | Mixed |
Closest papers by meaning
- Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits (NeurIPS 2025 · similarity 0.89)
- Unveiling Transformer Perception by Exploring Input Manifolds (NeurIPS 2025 · similarity 0.86)
- DePass: Unified Feature Attributing by Simple Decomposed Forward Pass (NeurIPS 2025 · similarity 0.86)
- A unified framework for establishing the universal approximation of transformer-type architectures (NeurIPS 2025 · similarity 0.86)
- Predictive Geometry of Hidden Trajectories in Transformers (NeurIPS 2026 · similarity 0.86)
- Small Singular Values Matter: A Random Matrix Analysis of Transformer Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
