Research map / Learning theory and algorithms / Prediction and decision losses
Regression-adjusted Monte Carlo Estimators for Shapley Values and Probabilistic Values
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Regression MSR, which uses any efficiently-attributable function (linear or XGBoost-style trees) as a control variate inside maximum sample reuse for probabilistic values, with cross-fitting for unbiasedness.
Problem
Estimating probabilistic values (Shapley, Banzhaf, semivalues) requires exponentially many value-function evaluations; existing Monte Carlo methods have high variance and regression-based methods (Kernel/Leverage SHAP) are biased and do not extend well to general probabilistic values.
Approach
Use a learned surrogate f of the value function as a variance-reduction control variate inside maximum sample reuse, exploiting linearity of probabilistic values (phi(v)=phi(f)+phi(v-f)); cross-fitting makes the estimator unbiased, and any function class with efficient probabilistic-value computation (linear, tree ensembles) can be used.
Key results
- Tree MSR achieves mean relative error 3.4x lower than Leverage SHAP and 11x lower than Permutation SHAP at m=40n for Shapley values.
- For general probabilistic values (beta Shapley, weighted Banzhaf), Tree MSR gives mean error up to 390x lower than the best prior estimator at m=40n.
- Linear MSR performs comparably to Leverage SHAP (about 17% higher average error) while being unbiased under cross-fitting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Tree MSR instead of Leverage SHAP | Shapley value estimation · Eight datasets, m=40n | relative squared ℓ2 error: 3.4× lower average error | Helped |
| Use Tree MSR instead of the best prior estimator | Beta Shapley and weighted Banzhaf estimation · Large datasets, m=40n | relative squared ℓ2 error: 390× lower average error | Helped |
| Replace the linear approximation with a tree-based approximation (ablation) | Regression MSR for Shapley values · Eight datasets, m=40n | relative squared ℓ2 error: 5.35e-04 -> 1.35e-04 | Helped |
| Replace the linear approximation with a tree-based approximation (ablation) | Regression MSR for beta Shapley and weighted Banzhaf values · Large datasets, m=40n | relative squared ℓ2 error: 8.48e-04 -> 1.53e-04 | Helped |
| Add cross-fitting to Tree MSR (ablation) | Tree MSR for probabilistic values · 112 dataset–weighting pairs | relative squared ℓ2 error: 90 of the 112 pairs; all-samples error typically 1.8× larger | Helped |
| Add cross-fitting to Linear MSR (ablation) | Linear MSR for probabilistic values · 112 dataset–weighting pairs | relative squared ℓ2 error: 53 of the 112 pairs | Mixed |
| Add cross-fitting to Tree MSR (ablation) | Tree MSR · All-samples comparison | runtime: 2.6--9.9× longer runtime | Hurt |
| Use Tree MSR at low sample budgets instead of Kernel SHAP | Shapley value estimation · Low sample regime | relative squared ℓ2 error | Hurt |
Surprising findings
- The all-samples (k=1) linear version reduces exactly to Leverage SHAP for Shapley values, since the correction term vanishes.
- Cross-fitting costs 2.6-9.9x runtime for trees but the all-samples tree version has about 1.8x larger error.
Limitations
- Performance depends on how well the learned model fits the value function; poor for unstructured data or small sample budgets.
- Tree MSR underperforms Kernel SHAP in the low-sample regime.
Experimental setup
Datasets: Adult Forest Fires Real Estate Bike Sharing Breast Cancer Independent NHANES Communities
Benchmarks: Adult Forest Fires Real Estate Bike Sharing Breast Cancer Independent NHANES Communities
Metrics: relative squared l2 error
Baselines: Permutation SHAP Kernel SHAP Leverage SHAP Kernel Banzhaf Proxy SPEX OFA WSL GELS ARM WeightedSHAP
Builds on: Leverage SHAP Kernel SHAP Maximum Sample Reuse XGBoost
Closest papers by meaning
- Provably Accurate Shapley Value Estimation via Leverage Score Sampling (ICLR 2025 · similarity 0.90)
- A Unified Framework for Provably Efficient Algorithms to Estimate Shapley Values (NeurIPS 2025 · similarity 0.89)
- One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently (NeurIPS 2024 · similarity 0.87)
- shapiq: Shapley Interactions for Machine Learning (NeurIPS 2024 · similarity 0.87)
- Amortized Linear-time Exact Shapley Value for Product-Kernel Methods (NeurIPS 2026 · similarity 0.86)
- Prediction via Shapley Value Regression (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.