Research map / Architectures and efficiency / Attention and transformers
On Identifiability in Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a learned linear projection with cosine nearest-neighbor lookup to recover input-token identity | BERT contextual embeddings · MRPC development set, last layer | token identifiability rate: 93% | Helped |
| Use cosine distance rather than L2 distance for nearest-neighbor identity recovery | BERT contextual embeddings · MRPC development set | token identifiability rate | Helped |
| Project raw attention out of the attention-value left null space | BERT attention explanations · Wikipedia samples, sequences shorter than 64 tokens | Pearson correlation between raw and effective attention: 1 | No clear effect |
| Use effective attention instead of raw attention to interpret the middle-layer [SEP] peak | BERT attention explanations · Wikipedia samples, middle layers | relevance of [SEP] attention to output computation | Helped |
| Increase transformer depth | BERT contextual embeddings · MRPC development set | same-position input contribution: 30.6% -> 14.4% -> 10.7% | Hurt |
| Increase transformer depth | BERT contextual embeddings · MRPC development set | tokens not primarily contributed by their same-position input: 18% -> 30% | Hurt |
| Train the layer-11 MLP with more training data | BERT token identity recovery · MRPC development set, layer 11 | token identifiability rate: 0.74, 0.8, 0.81, 0.82 | Helped |
| Use a larger MLP hidden layer and add dropout (ablation) | BERT token identity recovery · MRPC development set | training time | Hurt |
| Use an MLP instead of a linear model for identity recovery (ablation) | BERT contextual embeddings · MRPC development set | generalization error | No clear effect |
| Increase the MLP training data from 90% to 100% | BERT token identity recovery · MRPC development set, layer 11 | token identifiability rate: 0.81 -> 0.82 | Helped |
Closest papers by meaning
- Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding (NeurIPS 2025 · similarity 0.88)
- On the Emergence of Position Bias in Transformers (ICML 2025 · similarity 0.88)
- What are you sinking? A geometric approach on attention sink (NeurIPS 2025 · similarity 0.88)
- Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits (NeurIPS 2025 · similarity 0.88)
- On the Role of Attention Masks and LayerNorm in Transformers (NeurIPS 2024 · similarity 0.88)
- The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.