Research map / Learning theory and algorithms / Prediction and decision losses
Shaping Up SHAP: Enhancing Stability through Layer-Wise Neighbor Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows Kernel SHAP instability stems from its stochastic layer-wise neighbor sampling, proposes ST-SHAP that fills complete layers deterministically for full stability, and derives a fast Layer-1-only attribution method.
Problem
Kernel SHAP explanations are unstable: repeated runs with identical inputs yield significantly different feature attributions, reducing their reliability.
Approach
Replace Kernel SHAP's stochastic neighbor selection with deterministic layer-wise filling (dropping the full-enumeration condition), so that complete layers yield fully reproducible explanations; restricting to Layer 1 coalitions gives a closed-form, stable, fast attribution method in the LES family.
Key results
- ST-SHAP achieves a Jaccard stability of 1.0 when the budget yields complete layers, whereas Kernel SHAP remains unstable.
- ST-SHAP's fidelity (adherence) is comparable to Kernel SHAP across datasets, and often higher on Spambase.
- Layer-1-only ST-SHAP attributions achieve Kendall tau around 0.6-1.0 and R^2 often above 0.9 versus exact SHAP values on several datasets, and run up to three orders of magnitude faster than exact SHAP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Kernel SHAP's stochastic neighbor selection with layer-wise filling, restricting random sampling to the incomplete layer | Kernel SHAP (ST-SHAP) · Boston, Dry Bean, HELOC, Spambase and other evaluated datasets | Jaccard coefficient | Helped |
| Use budgets that enumerate only complete layers | ST-SHAP · Boston and other evaluated datasets | Jaccard coefficient: 1.0 | Helped |
| Replace Kernel SHAP neighbor selection with layer-wise filling | Kernel SHAP (ST-SHAP) · Boston, Dry Bean, HELOC, Spambase and other evaluated datasets | adherence to black-box predictions | No clear effect |
| Restrict neighbor selection to Layer 1 | ST-SHAP · Boston | Kendall tau with exact SHAP values: 0.95 | Helped |
| Restrict neighbor selection to Layer 1 | ST-SHAP · Dry Bean | R^2 with exact SHAP values: -0.07 | Mixed |
| Use only Layer 1 coalitions | ST-SHAP · evaluated datasets | computation time | Helped |
Surprising findings
- Layer-1 attributions are already fairly faithful to exact SHAP values despite using only 2M coalitions, with fidelity high even at small budgets.
- Kernel SHAP's stochastic layer switching, rather than the budget alone, is identified as the main source of instability.
Limitations
- Layer-1 attributions are a theoretical approximation whose alignment with exact SHAP values is only empirically shown, with weaker results on some models (e.g., Credit Card RF R^2 0.02).
- Stability is evaluated on a limited set of 10 instances per dataset with 20 repeated runs.
- Experiments run on tabular data with scikit-learn black-box models only; the paper reports no results on neural or high-dimensional inputs.
Experimental setup
Datasets: Boston Movie Default of Credit Card Clients Adult Dry Bean Spambase HELOC Wisconsin Diagnostic Breast Cancer
Benchmarks: Boston Movie Dry Bean Adult Credit Card Spambase HELOC WDBC
Metrics: Jaccard coefficient Variable Stability Index adherence (R^2 / accuracy) Kendall tau R^2 score execution time
Baselines: Kernel SHAP exact SHAP values
Builds on: Kernel SHAP SHAP LIME LES values
Closest papers by meaning
- SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism (NeurIPS 2025 · similarity 0.86)
- SHAP values via sparse Fourier representation (NeurIPS 2025 · similarity 0.86)
- On the Tractability of SHAP Explanations under Markovian Distributions (ICML 2024 · similarity 0.86)
- Provably Accurate Shapley Value Estimation via Leverage Score Sampling (ICLR 2025 · similarity 0.85)
- shapiq: Shapley Interactions for Machine Learning (NeurIPS 2024 · similarity 0.84)
- A Unified Framework for Provably Efficient Algorithms to Estimate Shapley Values (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.