Research map / Label-efficient and robust learning / Recognition and long-tail
Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that models are miscalibrated under explainability-specific perturbations, proves this biases global and local perturbation-based explanations, and proposes ReCalX, a perturbation-level-adaptive temperature scaling that improves calibration, explanation robustness, and remove-and-retrain fidelity.
Problem
Perturbation-based explanations rely on model predictions under out-of-distribution feature perturbations, where models are often miscalibrated, distorting feature importance estimates and making explanations unstable.
Approach
Derives a decomposition of perturbation-based predictive power showing calibration error under each subset perturbation undermines explanations, and proposes ReCalX: information-preserving temperature scaling with separate temperatures per perturbation-level bin, fit on perturbed validation samples.
Key results
- ReCalX reduces maximum KL calibration error by 75.1% to 92.3% on tabular MLP models and by 85.1% to 97.4% on ImageNet image models relative to uncalibrated models.
- On the Electricity dataset, removing the top three ReCalX-ranked features increased loss by 33% versus 24% for the original model's Shapley explanations.
- ReCalX lowers average and maximum sensitivity of LIME, Kernel SHAP and Feature Ablation explanations across ResNet50, DenseNet121, ViT and SigLIP on ImageNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive perturbation-specific temperature scaling instead of no calibration | MLP on Electricity with mean replacement · Electricity | maximum KL calibration error: 0.1534 -> 0.0163 | Helped |
| Use adaptive perturbation-specific temperature scaling instead of static temperature scaling | MLP on Electricity with mean replacement · Electricity | maximum KL calibration error: 0.1664 -> 0.0163 | Helped |
| Use adaptive perturbation-specific temperature scaling instead of no calibration | ResNet on Pol with mean replacement · Pol | maximum KL calibration error: 0.8633 -> 0.0910 | Helped |
| Use static temperature scaling instead of no calibration | ResNet on Pol with mean replacement · Pol | maximum KL calibration error: 0.8633 -> 1.0173 | Hurt |
| Use adaptive perturbation-specific temperature scaling instead of no calibration | ViT with zero-baseline perturbations · ImageNet | maximum KL calibration error: 0.2618 -> 0.0078 | Helped |
| Use static temperature scaling instead of no calibration | ViT with zero-baseline perturbations · ImageNet | maximum KL calibration error: 0.2618 -> 0.3057 | Hurt |
| Use adaptive perturbation-specific temperature scaling instead of no calibration | ResNet50 with blur perturbations · ImageNet | maximum KL calibration error: 0.4158 -> 0.0139 | Helped |
| Apply ReCalX before generating LIME explanations | ResNet50 explanations with zero-baseline perturbations · ImageNet | average sensitivity: 1.349 -> 1.190 | Helped |
| Apply ReCalX before generating Kernel SHAP explanations | SigLIP explanations with blur perturbations · ImageNet | maximum sensitivity: 3.493 -> 2.897 | Helped |
| Rank global features using ReCalX explanations rather than original explanations | MLP feature removal and retraining with Shapley Values · Electricity | original loss increase after removing top three features: 24% -> 33% | Helped |
Surprising findings
- Standard temperature scaling calibrated on unperturbed samples can worsen calibration under perturbations, e.g., increasing ViT's maximum error from 0.262 to 0.306.
- Calibration error under perturbation does not always grow with perturbation level; ResNet50 is worst at low perturbation levels.
Limitations
- Requires held-out validation samples and additional setup effort for recalibration.
- Theoretical bounds hold for bounded linear summary strategies; the general non-linear case is treated separately in the appendix.
- Image experiments rely on temperature scaling, which is specific to classification; regression is only covered in an appendix.
Experimental setup
Datasets: Electricity Covertype Credit Pol ImageNet ILSVRC2012
Benchmarks: Electricity Covertype Credit Pol ImageNet ILSVRC2012
Metrics: KL-divergence calibration error (average and maximum over pe average sensitivity maximum sensitivity remove-and-retrain loss increase
Baselines: Uncalibrated model Temperature Scaling
Builds on: temperature scaling Shapley values LIME
Closest papers by meaning
- Reliable Post hoc Explanations: Modeling Uncertainty in Explainability (NeurIPS 2021 · similarity 0.90)
- Use perturbations when learning from explanations (NeurIPS 2023 · similarity 0.89)
- GNNX-BENCH: Unravelling the Utility of Perturbation-based GNN Explainers through In-depth Benchmarking (ICLR 2024 · similarity 0.86)
- Evaluate with the Inverse: Efficient Approximation of Latent Explanation Quality Distribution (AAAI 2025 · similarity 0.85)
- Trust Regions for Explanations via Black-Box Probabilistic Certification (ICML 2024 · similarity 0.85)
- Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.