Research map / Label-efficient and robust learning / Recognition and long-tail
Benchmarking the Attribution Quality of Vision Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the in-domain single-deletion score (IDSDS), which aligns train and test domains via patch-deletion fine-tuning, and uses it to benchmark 23 attribution methods and how ImageNet backbone design choices affect attribution correctness.
Problem
Existing deletion-based protocols for evaluating attribution maps suffer from out-of-domain issues, class information leakage, and cannot fairly compare different models, so the effect of model design on attribution quality is underexplored.
Approach
Fine-tune each model with a patch-deletion augmentation (one random patch deleted in half the training images) so train and test domains align exactly, then score attributions by the Spearman correlation between patch attribution sums and logit drops from single-patch deletion, enabling both attribution-method and inter-model comparisons.
Key results
- BagNet-33 achieves an IDSDS of 0.797 on ResNet-50 comparison, far above the best model-agnostic method IG-U at 0.255.
- Raw attribution values outperform absolute ones under IDSDS (e.g., IG-U 0.255 vs IG-U abs. 0.125), contrary to prior incremental-deletion findings.
- Removing BN and bias terms improves IDSDS for almost all methods, and IDSDS decreases with network depth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BagNet-33 instead of a standard ResNet-50 with post-hoc attribution | ImageNet attribution evaluation · ImageNet-1000 evaluation split | IDSDS: 0.236 -> 0.797 | Helped |
| Use B-cos ResNet-50 instead of standard ResNet-50 post-hoc attribution | ImageNet attribution evaluation · ImageNet-1000 evaluation split | IDSDS: 0.236 -> 0.314 | Helped |
| Use a uniform baseline rather than a zero baseline for Integrated Gradients (ablation) | Integrated Gradients on ResNet-50 · ImageNet-1000 evaluation split | IDSDS: 0.196 -> 0.255 | Helped |
| Use a zero baseline rather than a uniform baseline for RISE (ablation) | RISE on ResNet-50 · ImageNet-1000 evaluation split | IDSDS: 0.065 -> 0.131 | Helped |
| Add SmoothGrad (ablation) | Attribution methods on ResNet-50 · ImageNet-1000 evaluation split | IDSDS: 0.07 -> -0.005; 0.196 -> 0.049; 0.236 -> 0.102 | Hurt |
| Use raw rather than absolute attribution values for IG-U (ablation) | IG-U on ResNet-50 · ImageNet-1000 evaluation split | IDSDS: 0.125 -> 0.255 | Helped |
| Remove batch normalization (ablation) | ResNet models · ImageNet-1000 evaluation split | IDSDS | Helped |
| Remove remaining bias terms in addition to batch normalization (ablation) | ResNet models without batch normalization · ImageNet-1000 evaluation split | IDSDS | Helped |
| Increase network depth (ablation) | ResNet models · ImageNet-1000 evaluation split | IDSDS | Hurt |
| Increase network width (ablation) | ResNet-50 · ImageNet-1000 evaluation split | IDSDS | Hurt |
| Compute attributions after the final softmax layer rather than before it (ablation) | ResNet-50 attribution methods · ImageNet-1000 evaluation split | IDSDS | Hurt |
| Increase the number of evaluated patches (ablation) | IDSDS evaluation protocol · ImageNet-1000 evaluation split | IDSDS | Hurt |
Surprising findings
- Raw attribution values outperform absolute values under the proposed protocol, contradicting prior incremental-deletion findings.
- Integrated Gradients with uniform baseline outperforms IG with zero baseline despite the zero baseline matching the patch interventions.
Limitations
- Requires fine-tuning each model with the deletion augmentation, adding computational cost and possibly altering model behavior.
- Patch-level granularity (P=16) only; does not assess attribution quality within patches.
- Considers only single-patch deletions rather than patch combinations.
Experimental setup
Datasets: ImageNet-1000 FunnyBirds
Benchmarks: ImageNet-1000 validation (IDSDS) FunnyBirds
Metrics: IDSDS (Spearman rank correlation) incremental-deletion score (IDS) top-1 accuracy mean absolute difference of softmax outputs
Baselines: incremental-deletion score (IDS) out-of-domain single-deletion score (SDS) FunnyBirds ROAR
Builds on: incremental-deletion protocol single-deletion protocol ROAR FunnyBirds
Closest papers by meaning
- One Wave To Explain Them All: A Unifying Perspective On Feature Attribution (ICML 2025 · similarity 0.88)
- Pixel-level Certified Explanations via Randomized Smoothing (ICML 2025 · similarity 0.88)
- How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations (ICLR 2025 · similarity 0.87)
- Less is More: Fewer Interpretable Region via Submodular Subset Selection (ICLR 2024 · similarity 0.87)
- Rethinking Robustness of Model Attributions (AAAI 2024 · similarity 0.87)
- Comprehensive Information Bottleneck for Unveiling Universal Attribution to Interpret Vision Transformers (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.