Research map / Architectures and efficiency / RNNs and normalization
Towards better understanding of gradient-based attribution methods for Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Occlusion-1 attribution instead of other methods | Attribution methods evaluated on image and text classification models · Image and text classification tasks | Sensitivity-1 | Helped |
| Use Integrated Gradients, DeepLIFT, or LRP instead of Occlusion-1 | Attribution methods on nonlinear models · Image and text classification tasks | Sensitivity-n for medium-size feature sets | Helped |
| Use Integrated Gradients rather than Occlusion-1 | Attribution of the MNIST CNN with Sigmoid nonlinearities · MNIST | target-output variation when removing more than about 20 pixels | Helped |
| Use Gradient * Input rather than other gradient-based methods | MNIST-MLP with Tanh · MNIST-MLP with Tanh | correlation with Occlusion-1 | Helped |
| Approximate Integrated Gradients with DeepLIFT (Rescale) | DeepLIFT attribution · Various tasks, especially simple models | correlation with Integrated Gradients | Helped |
| Apply DeepLIFT (Rescale) to a model with multiplicative interactions | DeepLIFT (Rescale) · IMDB-LSTM | Completeness | Hurt |
| Use epsilon-LRP with ReLU nonlinearities | epsilon-LRP · Networks using ReLU | equivalence to Gradient * Input | Helped |
| Use epsilon-LRP with Sigmoid or Softplus nonlinearities | epsilon-LRP · Networks using Sigmoid or Softplus | attribution meaningfulness | Hurt |
| Apply epsilon-LRP under the conditions of zero baseline, no additive biases, and nonlinearities crossing the origin | epsilon-LRP · Networks meeting those conditions | equivalence to DeepLIFT (Rescale) | Helped |
| Apply Gradient * Input, epsilon-LRP, Integrated Gradients, DeepLIFT, or Occlusion-1 to a model that behaves linearly | Attribution methods in the paper · IMDB-MLP | Sensitivity-n correlation | Helped |
| Use gradient-based methods on complex Inception V3 | Gradient-based attribution methods · ImageNet-compatible samples with Inception V3 | attribution-sign prediction accuracy | Hurt |
| Use a single-feature occlusion benchmark to identify individual important features | Occlusion-1 · Image and text classification tasks | Sensitivity-n as n increases | Hurt |
Closest papers by meaning
- Axiomatic Attribution for Deep Networks (ICML 2017 · similarity 0.91)
- Distilled Gradient Aggregation: Purify Features for Input Attribution in the Deep Neural Network (NeurIPS 2022 · similarity 0.91)
- Iterative Search Attribution for Deep Neural Networks (ICML 2024 · similarity 0.90)
- How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations (ICLR 2025 · similarity 0.88)
- MFABA: A More Faithful and Accelerated Boundary-Based Attribution Method for Deep Neural Networks (AAAI 2024 · similarity 0.88)
- On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.