Research map / Label-efficient and robust learning / Test-time adaptation
Debugging Tests for Model Explanations
Julius Adebayo, Michael Muelly, Ilaria Liccardi, Been Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| inject spurious background and check attributions vs ideal mask | feature attributions · spurious sky/bamboo background | SSIM-GT2: 0.78 to 0.98 | Improved |
| compare attributions of mislabelled vs correctly labelled training input | feature attributions · 10% flipped labels | SSIM: 0.73-0.99 | No clear effect |
| successively re-initialize VGG-16 weights from top layer | modified backprop methods (GBP, DConvNet, PNet, PAttribution, LRP-SPAF) · VGG-16 ImageNet | attribution change | No clear effect |
| compare attributions from out-of-domain model with in-domain model | Gradient-style attributions · Fashion MNIST to ImageNet VGG-16 | SSIM / rank correlation: SSIM 0.87 for VGrad, rank correlation 0.028 | Mixed |
| compare FMNIST attributions on MNIST model vs FMNIST model | Guided BackProp · FMNIST to MNIST model | rank correlation: SSIM 0.81, rank correlation 0.51 | Mixed |
| Pattern Attribution on out-of-domain VGG-16 | PAttribution · FMNIST to VGG-16 | SSIM: 0.77 -> 0.12 | Hurt |
| normal vs spurious model human ratings | human study · Likert recommendation, Gradient | median Likert score: 4 -> 2 | Hurt |
| normal vs spurious model human ratings | human study · Likert recommendation, Integrated Gradients | median Likert score: 3 -> 3 | No clear effect |
| mislabelled training examples shown to users | human study · Likert recommendation, Gradient | median Likert score: 4 -> 1 | Hurt |
Closest papers by meaning
- Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation (ICLR 2022 · similarity 0.87)
- Meaningfully debugging model mistakes using conceptual counterfactual explanations (ICML 2022 · similarity 0.87)
- HiBug: On Human-Interpretable Model Debug (NeurIPS 2023 · similarity 0.87)
- How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations (ICLR 2025 · similarity 0.84)
- Time Travel in LLMs: Tracing Data Contamination in Large Language Models (ICLR 2024 · similarity 0.84)
- Data-centric Prediction Explanation via Kernelized Stein Discrepancy (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
