Research map / Learning theory and algorithms / Prediction and decision losses
Improving Decision Sparsity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends the Sparse Explanation Value (SEV) decision-sparsity framework with multiple cluster-based and flexible references, tree-based SEV, and credibility-aware explanations, plus gradient- and search-based training to optimize decision sparsity.
Problem
Local decision sparsity: explanations for individual binary classifier decisions are often not sparse, close, or credible when the reference point is far from the query or lies in a low-density region.
Approach
Replaces the single population-mean reference of SEV with multiple auditable cluster-centroid references, allows small flexible shifts of references to lower model output, uses shortest-path computation on decision trees via stored negative-leaf paths, and optimizes models for low SEV via a differentiable surrogate loss or exhaustive Rashomon-set search.
Key results
- Cluster-based SEV improves closeness over single-mean SEV-1 and sometimes improves both sparsity and closeness (e.g., COMPAS, Diabetes, German Credit).
- Gradient-based optimization (SEV_All_Opt) reduces average SEV while preserving closeness and credibility; search-based TOpt achieves average SEV of 1.00 without performance loss.
- In the counterfactual comparison, SEV variants give the sparsest explanations, e.g., on Adult SEV-1 changes 1.18 features vs. 2.19 for DiCE and 14 for Growing Sphere.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use cluster-based references instead of a single population-mean reference | SEV with L2 logistic regression · Adult | features changed: 1.18±0.02 -> 1.34±0.02 | Hurt |
| Use cluster-based references instead of a single population-mean reference | SEV with L2 logistic regression · COMPAS | features changed: 1.22±0.02 -> 1.19±0.02 | Helped |
| Use cluster-based references instead of a single population-mean reference | SEV with L2 logistic regression · Diabetes | features changed: 1.63±0.01 -> 1.28±0.02 | Helped |
| Use cluster-based references instead of a single population-mean reference | SEV with L2 logistic regression · FICO | features changed: 2.76±0.02 -> 2.21±0.02 | Helped |
| Use cluster-based references instead of a single population-mean reference | SEV with L2 logistic regression · Adult | median log-likelihood: -24752.12±452.47 -> 156.88±59.67 | Helped |
| Use cluster-based references instead of a single population-mean reference | SEV with L2 logistic regression · COMPAS | median log-likelihood: 14.65±0.32 -> 14.41±0.05 | Hurt |
| Use cluster-based references instead of a single population-mean reference | SEV with L2 logistic regression · Diabetes | median log-likelihood: 309.56±15.32 -> 320.71±14.79 | Helped |
| Use cluster-based references instead of a single population-mean reference | SEV with L2 logistic regression · FICO | median log-likelihood: -20.11±0.32 -> -19.32±0.21 | Helped |
| Add flexible reference adjustment (ablation) | Cluster-based SEV · Multiple datasets and models | SEV | Helped |
| Add flexible reference adjustment (ablation) | Cluster-based SEV · Multiple datasets and models | credibility | No clear effect |
| Add credibility-based continued traversal of the SEV hypercube (ablation) | SEV explanations · Negative population | credibility | Helped |
| Add the SEV penalty to the training objective (ablation) | Gradient-based SEV optimization · FICO | SEV | Helped |
Surprising findings
- Cluster-based SEV sometimes achieves lower SEV and better closeness simultaneously than the single-mean reference.
- Moving references closer to the decision boundary can increase average SEV (e.g., L2LR mean SEV rises from 2.76 to 4.95 for the moved reference).
Limitations
- Causal relationships among features may invalidate transitions on the SEV hypercube.
- Choosing the stopping threshold for credibility-based walking is not easy to determine.
- Cluster-based and flexible references require auditable choices and tuning of clustering and flexibility parameters.
Experimental setup
Datasets: UCI Adult Income FICO HELOC UCI German Credit MIMIC-III COMPAS Diabetes 130-US hospitals News Headlines survey
Benchmarks: FICO Adult COMPAS German Credit MIMIC-III Diabetes Headlines
Metrics: SEV (average number of feature changes) median L-infinity closeness log-likelihood credibility under Gaussian mixture model test accuracy
Baselines: SEV with single mean reference (SEV-1) Watcher REVISE Growing Spheres DiCE CART C4.5 GOSDT ExpO
Builds on: Sparse Explanation Value TreeFARMS ExpO soft k-means CARLA
Closest papers by meaning
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.86)
- Local vs. Global Interpretability: A Computational Complexity Perspective (ICML 2024 · similarity 0.84)
- Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again (NeurIPS 2026 · similarity 0.83)
- Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention (AAAI 2024 · similarity 0.83)
- Efficient and Accurate Explanation Estimation with Distribution Compression (ICLR 2025 · similarity 0.83)
- Near-Optimal Decision Trees in a SPLIT Second (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.