Research map / Probabilistic and scientific modeling / Molecules and proteins
X-Hacking: The Threat of Misguided AutoML
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces X-hacking, the manipulation of SHAP-based explanations by searching a Rashomon set of accurate AutoML pipelines, and shows post-hoc cherry-picking and Bayesian multi-objective directed search can find defensible models that flip explanations.
Problem
XAI metrics such as SHAP values can be manipulated by analysts who search over many defensible, similarly accurate models (model multiplicity) to obtain a pre-specified explanation, undermining credibility and reproducibility of XAI.
Approach
Frames explanation hacking as p-hacking on XAI metrics: uses AutoML (auto-sklearn search space) either post-hoc to cherry-pick from a confirmatory Rashomon set, or ad-hoc via Bayesian multi-objective optimisation (Optuna MOTPE, Ray Tune) trading off accuracy against mean absolute SHAP of top features, and shows redundancy among features governs vulnerability.
Key results
- Bayesian multi-objective optimisation reached the overall minimum mean absolute SHAP found by random sampling about 3 hours earlier on average (~3x faster) for features susceptible to X-hacking.
- Using directed search, a defensible confirmatory model was found in under 6 minutes for 14 of 23 datasets, roughly 10x faster than post-hoc cherry-picking (which needed an hour or more for many datasets).
- 20 of 92 targeted features yielded no defensible models under either method, indicating robustness to X-hacking.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bayesian multi-objective optimization instead of random sampling to search for defensible models with lower mean absolute SHAP | Custom AutoML X-hacking · Features susceptible to X-hacking across OpenML CC-18 datasets | time to reach the random-sampling minimum mean absolute SHAP: 3 hours earlier | Helped |
| Use Bayesian multi-objective optimization instead of random sampling to search for defensible models with lower mean absolute SHAP | Custom AutoML X-hacking · Majority of the 92 selected features | search speed: 3 times faster | Helped |
| Use directed search instead of post-hoc cherry-picking | X-hacking · Datasets and features vulnerable to X-hacking | time to find a confirmatory defensible model: 10x faster | Helped |
| Search pipelines that use redundant information among features | SHAP explanations · Simulated data with redundant features | ability to flip SHAP values without loss of predictive accuracy | Helped |
| Remove information redundancy among features | SHAP explanations · Simulated data with independent features | ability to manipulate SHAP values without changing predictive accuracy | Hurt |
| Use off-the-shelf AutoML to vary modeling choices while retaining at least baseline accuracy | Post-hoc X-hacking · cardiac-disease, height feature | SHAP dependence slope | Mixed |
| Use off-the-shelf AutoML to vary modeling choices while retaining at least baseline accuracy | Post-hoc X-hacking · cardiac-disease, top three features | change in mean absolute SHAP | No clear effect |
| Apply directed search to features with no defensible solutions in either optimizer | Custom AutoML X-hacking · 20/92 selected features | finding a defensible model: 20/92 | No clear effect |
Surprising findings
- Post-hoc AutoML cherry-picking failed to topple the baseline top feature for about a third of datasets within the time budget.
- Without information redundancy among features, SHAP values cannot be flipped without losing accuracy.
Limitations
- Proof-of-concept only; SHAP metrics used, with other XAI metrics (p-values, fairness) left to future work.
- Defensibility approximated by accuracy exceeding a baseline rather than peer-review justification.
- SHAP approximated with KernelExplainer on small background (50) and test (100) samples.
Experimental setup
Datasets: OpenML-CC18 binary classification datasets (23) simulated collinear-feature datasets
Benchmarks: OpenML-CC18 binary tasks (23 datasets)
Metrics: accuracy mean absolute SHAP of top features time to first defensible model cumulative minimum SHAP
Baselines: random sampling random forest baseline
Builds on: auto-sklearn Optuna Ray Tune SHAP p-hacking
Closest papers by meaning
- Rethinking XAI Evaluation: A Human-Centered Audit of Shapley Benchmarks in High-Stakes Settings (NeurIPS 2026 · similarity 0.87)
- Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics (NeurIPS 2024 · similarity 0.85)
- F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI (ICLR 2025 · similarity 0.85)
- Position: Explain to Question not to Justify (ICML 2024 · similarity 0.84)
- Towards Trustable SHAP Scores (AAAI 2025 · similarity 0.84)
- Evaluate with the Inverse: Efficient Approximation of Latent Explanation Quality Distribution (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.