Research map / Architectures and efficiency / Attention and transformers
XAI for Transformers: Better Explanations through Conservative Propagation
Ameen Ali, Thomas Schnake, Oliver Eberle, Grégoire Montavon, Klaus-Robert Müller, Lior Wolf
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LRP-style detaching of attention gating and LayerNorm rescaling (AH+LN) over plain Gradient x Input | Transformer and Graphormer explanations · Perturbation activation/pruning tasks across 9 datasets | AUAC / AU-MSE: e.g. SST-2 AUAC .847 -> .908; SST-2 AU-MSE 2.10 -> 1.56 | Improved |
| Detach attention gating terms only (AH rule) (ablation) | Gradient x Input on Transformer · SST-2 | AUAC: .847 -> .855 | Improved |
| Detach LayerNorm rescaling only (LN rule) (ablation) | Gradient x Input on Transformer · SST-2 | AUAC: .847 -> .907 | Improved |
| Detach LayerNorm rescaling only (LN rule) (ablation) | Gradient x Input on Transformer · SST-2 | AU-MSE: 2.10 -> 1.78 | Improved |
| Detach attention gating terms only (AH rule) | Gradient x Input on Transformer · MNIST superpixels (Graphormer) | AU-MSE: 18.06 -> 18.03 | No clear effect |
| Detach attention gating terms only (AH rule) | Gradient x Input on BACE Graphormer · BACE | AUAC: .646 -> .645 | No clear effect |
| Detach attention gating terms only (AH rule) | Gradient x Input on BACE Graphormer · BACE | AU-MSE: 2.06 -> 2.08 | Hurt |
| Detach attention gating terms only (AH rule) | Gradient x Input on IMDB · IMDB | AUAC: .920 -> .911 | Hurt |
| Detach attention gating terms only (AH rule) | Gradient x Input on IMDB · IMDB | AU-MSE: 0.87 -> 0.77 | Improved |
| Detach LayerNorm rescaling only (LN rule) vs GI on BACE | Gradient x Input on BACE Graphormer · BACE | AUAC: .646 -> .702 | Improved |
| Gradient x Input baseline vs raw attention and GAE baselines | Explanation methods · Perturbation tasks across datasets | AUAC: e.g. IMDB GI .920 vs Rollout .738 | Improved |
| Use Gradient x Input with unmodified gradients (no conservation fix) in LayerNorm/attention | Gradient x Input on Transformer · SST-2 conservation check | conservation (sum of relevance vs output) | Hurt |
Closest papers by meaning
- Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability (NeurIPS 2025 · similarity 0.87)
- AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers (ICML 2024 · similarity 0.87)
- In-context Convergence of Transformers (ICML 2024 · similarity 0.87)
- What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis (ICLR 2025 · similarity 0.87)
- How Transformers Learn Structured Data: Insights From Hierarchical Filtering (ICML 2025 · similarity 0.87)
- Gnothi Seauton: Empowering Faithful Self-Interpretability in Black-Box Transformers (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
