Research map / Learning theory and algorithms / Prediction and decision losses

Position: Rethinking Explainable Machine Learning as Applied Statistics

ICML 2025 · Learning theory and algorithms › Prediction and decision losses · approach: Provable post-hoc explanations

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that post-hoc explainable machine learning should be treated as applied statistics of high-dimensional functions, with explanations as "statistics of functions" whose value depends on having a clear interpretation.

Problem

Explainable machine learning lacks agreed meaning for its basic terms (explanation, interpretability), and many explanation algorithms are proposed without specifying what intuitive question they answer, leading to misinterpretation.

Approach

Formalize explanations as statistics of functions (functionals mapping a function, distribution and data point to a vector), analogous to statistics of distributions and datasets, and argue that the key question is whether a statistic has a clearly specified interpretation relating it to an intuitive concept.

Key results

• Defines a statistic of a function as a functional F: F(R^d) x P(R^d) x R^d -> R^k and argues most local post-hoc explanations (SHAP, LIME, LRP, Grad-CAM, counterfactuals) fit this definition.
• Argues many post-hoc explanation methods lack a clear interpretation, with feature attribution methods showing divergence rather than convergence across formalizations.
• Recommends that explanation methods be designed for specific questions, that benchmarks probe empirical properties but do not supply interpretations, and that explanations are unlikely to replace fairness or robustness statistics.

Surprising findings

• Argues that benchmark datasets are generally unable to provide interpretations for explanation outputs, analogous to how one would not benchmark the interpretive performance of mean versus median.

Limitations

• No empirical evaluation; argument rests on conceptual analysis and examples from the literature.
• Scope restricted chiefly to post-hoc explanation algorithms; mechanistic interpretability only briefly discussed.

Experimental setup

Builds on: SHAP LIME Layer-wise relevance propagation Grad-CAM

Closest papers by meaning

• Local vs. Global Interpretability: A Computational Complexity Perspective (ICML 2024 · similarity 0.85)
• On the Importance of Application-Grounded Experimental Design for Evaluating Explainable ML Methods (AAAI 2024 · similarity 0.85)
• Even-if Explanations: Formal Foundations, Priorities and Complexity (AAAI 2025 · similarity 0.84)
• Efficient and Accurate Explanation Estimation with Distribution Compression (ICLR 2025 · similarity 0.84)
• Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? (ICLR 2025 · similarity 0.83)
• Data-centric Prediction Explanation via Kernelized Stein Discrepancy (ICLR 2025 · similarity 0.83)