Research map / Learning theory and algorithms / Prediction and decision losses
Saliency strikes back: How filtering out high frequencies improves white-box explanations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FORGrad, which low-pass filters the input gradient with a per-architecture, per-method cut-off frequency chosen to maximize faithfulness, so white-box attribution methods become competitive with black-box methods at much lower cost.
Problem
White-box attribution methods (gradient-based) are computationally cheap but less faithful than black-box methods; the paper attributes this gap to high-frequency artifacts inherited from the input gradient.
Approach
Filter the input gradient with a Fourier low-pass filter whose cut-off frequency is tuned per network architecture and attribution method to maximize faithfulness on training images; the authors trace the high-frequency content to max-pooling and striding in CNNs.
Key results
- FORGrad consistently raises faithfulness of white-box methods on ResNet50; e.g., Saliency rises from 0.18 to 0.27-0.28 and SmoothGrad from 0.23 to 0.37.
- After FORGrad, white-box methods occupy at least 4 of the top-5 global rankings on ResNet50 under the aggregate (F + muF) - S score.
- Filtering the gradient preserves first-order approximation error, while zero-gradient control gives much higher error, supporting that high frequencies are non-informative.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply FORGrad gradient filtering to Saliency (ablation) | Saliency on ResNet50 · ImageNet validation | Faithfulness: 0.18 -> 0.28 | Helped |
| Apply FORGrad gradient filtering to GuidedBackprop (ablation) | GuidedBackprop on ResNet50 · ImageNet validation | Faithfulness: 0.31 -> 0.35 | Helped |
| Apply FORGrad gradient filtering to GradInput (ablation) | GradInput on ResNet50 · ImageNet validation | Faithfulness: 0.2 -> 0.26 | Helped |
| Apply FORGrad gradient filtering to Integrated Gradients (ablation) | Integrated Gradients on ResNet50 · ImageNet validation | Faithfulness: 0.24 -> 0.31 | Helped |
| Apply FORGrad gradient filtering to SmoothGrad (ablation) | SmoothGrad on ResNet50 · ImageNet validation | Faithfulness: 0.23 -> 0.37 | Helped |
| Apply FORGrad gradient filtering to VarGrad (ablation) | VarGrad on ResNet50 · ImageNet validation | Faithfulness: 0.36 -> 0.35 | Hurt |
| Apply FORGrad gradient filtering to SquareGrad (ablation) | SquareGrad on ResNet50 · ImageNet validation | Faithfulness: 0.36 -> 0.36 | No clear effect |
| Apply FORGrad gradient filtering to Saliency (ablation) | Saliency on ConvNeXt · ImageNet validation | Faithfulness: -0.01 -> 0.06 | Helped |
| Apply FORGrad gradient filtering to Integrated Gradients (ablation) | Integrated Gradients on ConvNeXt · ImageNet validation | Faithfulness: 0.11 -> 0.07 | Hurt |
| Apply FORGrad gradient filtering to Integrated Gradients (ablation) | Integrated Gradients on ViT · ImageNet validation | Faithfulness: 0.19 -> 0.21 | Helped |
| Replace MaxPooling with AveragePooling (ablation) | ResNet50 gradients · Gradient frequency profile | High-frequency power | Helped |
| Set MaxPooling stride to 1 instead of greater than 1 (ablation) | VGG16 gradients · Gradient frequency profile | High-frequency content | Helped |
Surprising findings
- Max-pooling layers create checkerboard high-frequency artifacts in input gradients, and an untrained ResNet50 shows the same effect, so the artifact is architectural rather than learned.
- The mu-Fidelity metric is biased toward high-frequency attribution maps, so its scores changed little with FORGrad.
Limitations
- Benefit is smaller for ViT, which lacks max-pooling, and the authors note filtering may not help networks with inherently smooth gradients.
- Gains on the mu-Fidelity metric are limited, and the metric favors high-frequency maps, so evaluation is only partially reliable.
- Cut-off frequency must be tuned per architecture and attribution method using faithfulness on training images.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet validation subset (1,280 images)
Metrics: faithfulness (insertion minus deletion) mu-Fidelity sensitivity (stability) computation time
Baselines: Saliency Guided Backprop Gradient x Input Integrated Gradients SmoothGrad VarGrad SquareGrad Grad-CAM Grad-CAM++ Occlusion HSIC Sobol
Builds on: Saliency SmoothGrad VarGrad SquareGrad Grad-CAM
Closest papers by meaning
- How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations (ICLR 2025 · similarity 0.86)
- On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box (ICML 2024 · similarity 0.86)
- Selective Explanations (NeurIPS 2024 · similarity 0.86)
- Efficient and Accurate Explanation Estimation with Distribution Compression (ICLR 2025 · similarity 0.85)
- GEFA: A General Feature Attribution Framework Using Proxy Gradient Estimation (ICML 2025 · similarity 0.84)
- Local vs. Global Interpretability: A Computational Complexity Perspective (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.