Research map / Label-efficient and robust learning / Recognition and long-tail
On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GEEX, a black-box attribution method that estimates gradients via query-only search-distribution sampling integrated along a path from a baseline to the explicand, and reports it beats black-box baselines and approaches IG on MNIST, Fashion-MNIST and ImageNet.
Problem
Gradient-based attribution methods need internal model access, which is often impractical for safety reasons, while existing black-box explainers (LIME, RISE) give coarse, fuzzy attributions.
Approach
Replace the true gradient in Integrated Gradients with a search-distribution gradient estimator (location-family Gaussian with mirror sampling and merged path/sample sums), so attributions require only query access; proves Completeness, Sensitivity, Insensitivity/Missingness, Implementation Invariance and Linearity.
Key results
- On MNIST and Fashion-MNIST, GEEX reaches the highest normalized AOPC overall (e.g., 0.9466 on MNIST baseline replacement) and beats all black-box methods, slightly exceeding IG in these simple settings.
- On ImageNet with InceptionV3, GEEX (0.7952 baseline replacement, 0.9058 Gaussian) surpasses other black-box explainers but falls behind IG (0.8805 baseline replacement), with convergence toward IG as queries increase.
- Raw gradient estimation (GE) and SmoothGrad underperform on ImageNet, with GE scoring 0.3806 under baseline replacement, below random deletion-level reference in that setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GEEX instead of RISE | CNN · MNIST (Baseline replacement) | normalized AOPC: 0.9101 -> 0.9466 | Helped |
| Use GEEX instead of RISE | CNN · MNIST (Gaussian replacement) | normalized AOPC: 0.8896 -> 0.9486 | Helped |
| Use GEEX instead of RISE | CNN · Fashion-MNIST (Baseline replacement) | normalized AOPC: 0.8931 -> 0.9350 | Helped |
| Use GEEX instead of RISE | CNN · Fashion-MNIST (Gaussian replacement) | normalized AOPC: 0.8708 -> 0.9362 | Helped |
| Use GEEX instead of RISE | InceptionV3 · ImageNet (Baseline replacement) | normalized AOPC: 0.7659 -> 0.7952 | Helped |
| Use GEEX instead of RISE | InceptionV3 · ImageNet (Gaussian replacement) | normalized AOPC: 0.8699 -> 0.9058 | Helped |
| Use GEEX instead of IG | CNN · MNIST (Baseline replacement) | normalized AOPC: 0.9434 -> 0.9466 | Helped |
| Use GEEX instead of IG | CNN · Fashion-MNIST (Baseline replacement) | normalized AOPC: 0.9275 -> 0.9350 | Helped |
| Use GEEX instead of IG | InceptionV3 · ImageNet (Baseline replacement) | normalized AOPC: 0.8805 -> 0.7952 | Hurt |
| Use GEEX instead of IG | InceptionV3 · ImageNet (Gaussian replacement) | normalized AOPC: 0.9155 -> 0.9058 | Hurt |
Surprising findings
- GEEX outperforms the white-box IG on the simpler MNIST and Fashion-MNIST settings, attributed to a smoother path-integral approximation from merging sampling and integration.
- Raw gradient estimation and SmoothGrad fall below random deletion on ImageNet under baseline replacement, indicating gradient saturation hurts them.
Limitations
- Estimation variance grows with input dimensionality, so ImageNet results lag IG and need more queries.
- Mask smoothing breaks the independent-feature sampling assumption, trading correctness for usefulness on high-dimensional inputs.
- Hyperparameter allocation between interpolation steps and sample count is nontrivial; feature space decomposition via Linearity is not developed.
Experimental setup
Datasets: MNIST Fashion-MNIST ImageNet
Benchmarks: MNIST Fashion-MNIST ImageNet
Metrics: normalized AOPC (evaluation via deletion) qualitative saliency comparison
Baselines: SmoothGrad Integrated Gradients LIME RISE GE (raw gradient estimator) Random deletion
Builds on: Integrated Gradients SmoothGrad LIME RISE evolution strategies gradient estimation
Closest papers by meaning
- GEFA: A General Feature Attribution Framework Using Proxy Gradient Estimation (ICML 2025 · similarity 0.93)
- What You See is What You Classify: Black Box Attributions (NeurIPS 2022 · similarity 0.87)
- How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations (ICLR 2025 · similarity 0.86)
- Saliency strikes back: How filtering out high frequencies improves white-box explanations (ICML 2024 · similarity 0.86)
- Provably Better Explanations with Optimized Aggregation of Feature Attributions (ICML 2024 · similarity 0.85)
- Selective Explanations (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.