Research map / Adversarial robustness and security / Certified and empirical robustness
Interpretation of Neural Networks Is Fragile
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply an iterative top-k or mass-center attack instead of random sign perturbation | ImageNet feature-importance maps from simple gradients, integrated gradients, and DeepLIFT · ImageNet | top-1000 intersection and rank correlation | Hurt |
| Randomly perturb each pixel by ±ε | ImageNet feature-importance maps · ImageNet | top-1000 salient-pixel overlap: less than 30% overlap | Hurt |
| Use a mass-center attack rather than a top-k attack | Feature-importance maps · ImageNet | perceptual change in feature-importance maps | Helped |
| Use integrated gradients rather than simple gradients or DeepLIFT | Feature-importance methods · ImageNet and CIFAR-10 | robustness to interpretation attack | Helped |
| Apply a gradient sign attack to the test image | Influence-function explanations · Roses vs. sunflowers test set | top-5 influential training images retained: 2 of the top 5 | Hurt |
| Apply a gradient sign attack rather than random perturbation | Influence-function explanations · Roses vs. sunflowers test set | rank correlation | Hurt |
| Randomly perturb the test image | Influence-function explanations · Roses vs. sunflowers test set | rank correlation: 0.8 (ε ≈ 10) | Hurt |
| Replace ReLU with Softplus and retrain (ablation) | CIFAR-10 CNN · CIFAR-10 test | accuracy: 73% -> 73% | No clear effect |
| Use the feature-importance targeted attack to move saliency to a chosen image region | Simple gradients, integrated gradients, and DeepLIFT · ImageNet | semantic focus of saliency map | Helped |
| Apply small input perturbations to the interpretation | CIFAR-10 feature-importance maps · CIFAR-10 | robustness to interpretation attack | Mixed |
Closest papers by meaning
- Proper Network Interpretability Helps Adversarial Robustness in Classification (ICML 2020 · similarity 0.90)
- Evaluations and Methods for Explanation through Robustness Analysis (ICLR 2021 · similarity 0.88)
- Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing Their Input Gradients (AAAI 2018 · similarity 0.88)
- Theoretical Understanding of Learning from Adversarial Perturbations (ICLR 2024 · similarity 0.88)
- Wide Two-Layer Networks can Learn from Adversarial Perturbations (NeurIPS 2024 · similarity 0.86)
- Contrasting Adversarial Perturbations: The Space of Harmless Perturbations (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.