Research map / Adversarial robustness and security / Certified and empirical robustness
Rethinking Robustness of Model Attributions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes locality-sensitive (LENS) and diversity-based metrics for attributional robustness, arguing that top-k intersection overpenalizes minor local attribution shifts and that adversarial/IG-norm training helps attributions on small datasets but less on ImageNet.
Problem
Existing attributional robustness metrics (top-k intersection, Spearman's rho, Kendall's tau) penalize even one-pixel local shifts in attribution, making random perturbations look like strong attacks, and attributions concentrate on small regions even when multiple important parts exist.
Approach
Extend top-k intersection and rank correlations with neighborhood (window-based) matching so that locally shifted attributions count as similar (LENS-top-k, LENS-Spearman, LENS-Kendall), and add a greedy diverse top-k pixel selection (top-k-div) that prevents attributions from clustering in one region.
Key results
- On MNIST and Fashion MNIST, 1-LENS-prec@k (above 90%) is roughly twice top-k intersection (near 40%) under the top-k attack on naturally trained IG.
- On a Flower example, top-k intersection of 0.14 rises to 2-LENS-recall@k-div of 0.95 after combining locality and diversity.
- In a human survey on Flower images, 70.37% of participants agreed with the 3-LENS-prec@k metric over top-k intersection (29.63%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace top-k intersection with 1-LENS-prec@k | IG attribution robustness evaluation · MNIST and Fashion MNIST, top-t attack | attribution robustness: roughly 2x higher (above 90%) compared to top-k intersection (near 40%) | Helped |
| Increase locality window size w | w-LENS-prec@k · MNIST, Fashion MNIST, GTSRB, and Flower | attribution robustness | Helped |
| Add diversity to top-k pixel selection and use locality-sensitive evaluation | IG on Flower · Flower, top-k attributional attack | attribution robustness: 0.14 to 0.95 | Helped |
| Use 3-LENS-recall@k instead of top-k intersection | Attribution methods on ImageNet · ImageNet, random sign perturbation | attribution robustness | Helped |
| Use 3-LENS-recall@k-div instead of 3-LENS-recall@k | Natural-training attribution methods on ImageNet · ImageNet, random sign perturbation | attribution robustness | Helped |
| Train with PGD instead of natural training | Simple Gradient attribution · ImageNet, random sign perturbation | top-k intersection: 0.3825 -> 0.1725 | Hurt |
| Train with PGD instead of natural training | IG attribution · ImageNet, random sign perturbation | 3-LENS-recall@k: 0.7148 -> 0.7335 | Helped |
| Train with PGD instead of natural training | DeepLIFT attribution · ImageNet, random sign perturbation | 3-LENS-recall@k-div: 0.8504 -> 0.8977 | Helped |
| Train with PGD instead of natural training | DeepLIFT attribution · ImageNet, random sign perturbation | 3-LENS-recall@k: 0.7641 -> 0.9746 | Helped |
| Construct the attack using a 1-LENS-prec@k objective instead of top-k intersection | Attributional attack · Naturally trained models | top-k intersection | Hurt |
| Compare random and universal random perturbations | IG attribution robustness under existing metrics · GTSRB and Flower | top-k intersection: GTSRB: 0.8216 vs 0.9293; Flower: 0.8202 vs 0.9344 | Mixed |
| Use 1-smoothed attribution maps for rank-correlation evaluation | IG attribution robustness · MNIST, Fashion MNIST, GTSRB, and Flower | Spearman's rho and Kendall's tau | Helped |
Surprising findings
- Modifying the top-k attack to optimize 1-LENS-prec@k yields a weaker attack when measured by top-k intersection.
- A single universal random perturbation can be a more effective attributional attack than independent per-image random perturbations under top-k intersection.
Limitations
- Theoretical understanding of locality-sensitive metrics is incomplete and stronger attacks for these metrics remain open.
- The advantage of adversarially trained models for attributional robustness shrinks on larger datasets such as ImageNet (random sign attack).
- Experiments use a limited set of small datasets and a few architectures.
Experimental setup
Datasets: MNIST Fashion MNIST GTSRB Flower ImageNet
Benchmarks: MNIST Fashion MNIST GTSRB Flower ImageNet
Metrics: top-k intersection LENS-prec@k LENS-recall@k LENS-prec@k-div LENS-recall@k-div Spearman's rho Kendall's tau LENS-Spearman LENS-Kendall
Baselines: top-k intersection Spearman's rho Kendall's tau PGD-trained models IG-SUM-NORM trained models
Builds on: top-k attributional attack PGD adversarial training IG-SUM-NORM regularization Integrated Gradients
Closest papers by meaning
- On the Robustness of Removal-Based Feature Attributions (NeurIPS 2023 · similarity 0.91)
- Enhanced Regularizers for Attributional Robustness (AAAI 2021 · similarity 0.89)
- A Practical Upper Bound for the Worst-Case Attribution Deviations (CVPR 2023 · similarity 0.89)
- Pixel-level Certified Explanations via Randomized Smoothing (ICML 2025 · similarity 0.88)
- Benchmarking the Attribution Quality of Vision Models (NeurIPS 2024 · similarity 0.87)
- Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.