Research map / Probabilistic and scientific modeling / Molecules and proteins
OrdShap: Feature Position Importance for Sequential Black-Box Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OrdShap, a local attribution method for sequential black-box models that disentangles feature value importance (OrdShap-VI) from feature position importance (OrdShap-PI), linked to Sanchez-Bergantiños values.
Problem
Existing feature attribution methods for sequential models assume a fixed feature ordering, conflating a feature's value effect with its positional effect on predictions.
Approach
Extends Shapley-style coalition attributions with a positional characteristic function that permutes features to positions before ablation, yielding a feature-by-position OrdShap matrix whose row-averages (OrdShap-VI) equal Shapley values of an averaged game and whose linear slope over position (OrdShap-PI) measures positional effect; connects the VI term to the Sanchez-Bergantiños value.
Key results
- On IncAUC for value importance, OrdShap-VI achieves the best score on all five model/dataset settings (e.g., 0.913 on EICU-LOS vs. 0.906 for the next best, DeepLIFT).
- On ExcAUC, OrdShap-VI is best on four of five settings, with KernelSHAP best on IMDB (0.766 vs. 0.779 for OrdShap-VI).
- The least-squares OrdShap algorithm takes about 29 s per sample on EICU-LOS, versus 0.69 s for KernelSHAP; the sampling variant takes up to about 2470 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OrdShap-VI instead of competing explainers | feature attribution on EICU-LOS, EICU-Mort, MIMICIII-LOS, MIMICIII-Mort, and IMDB · EICU-LOS, EICU-Mort, MIMICIII-LOS, MIMICIII-Mort, and IMDB | Inclusion AUC: 0.906 -> 0.913; 0.804 -> 0.809; 0.898 -> 0.899; 0.859 -> 0.862; 0.863 -> 0.866 | Helped |
| Use OrdShap-VI instead of the strongest baseline | feature attribution on EICU-LOS, EICU-Mort, MIMICIII-LOS, and MIMICIII-Mort · EICU-LOS, EICU-Mort, MIMICIII-LOS, and MIMICIII-Mort | Exclusion AUC: 0.614 -> 0.573; 0.727 -> 0.724; 0.795 -> 0.788; 0.481 -> 0.472 | Helped |
| Use OrdShap-VI instead of KernelSHAP | feature attribution on IMDB · IMDB | Exclusion AUC: 0.766 -> 0.779 | Hurt |
| Permute features according to OrdShap-PI attributions | sequential-model predictions · EICU-LOS and other evaluated models | predicted-class probability | Mixed |
| Permute features according to traditional attribution methods rather than OrdShap-PI | sequential-model predictions · EICU-LOS and other evaluated models | predicted-class probability | Hurt |
| Use OrdShap least-squares approximation instead of sampling approximation | OrdShap · EICU-LOS, EICU-Mort, MIMICIII-LOS, MIMICIII-Mort, and IMDB | execution time (seconds): 29.29 (1.30) vs 2470.73 (685.0); 5.53 (0.30) vs 984.03 (264.0); 27.33 (4.41) vs 1369.00 (721.7); 5.49 (1.23) vs 565.51 ( | Helped |
Surprising findings
- Model predictions on IMDB remain relatively flat when sentences are permuted, suggesting DistilBERT is largely insensitive to sentence order for sentiment.
Limitations
- OrdShap is computationally expensive, requiring sums over 2^d coalitions and d! permutations, and is slower than competing methods.
- OrdShap-PI lacks a dedicated axiomatization; only the VI component is linked to the Sanchez-Bergantiños value.
- Assumes each feature can be permuted to any position, requiring an extension for grouped or irregular-time sequences.
Experimental setup
Datasets: MIMIC-III eICU IMDB
Benchmarks: MIMIC-III eICU IMDB synthetic dataset
Metrics: Inclusion AUC Exclusion AUC position importance permutation curve (change in predicted p execution time
Baselines: KernelSHAP LIME LOCO Integrated Gradients DeepLIFT Random TimeSHAP
Builds on: KernelSHAP Sanchez-Bergantiños value permutation sampling Shapley approximation
Closest papers by meaning
- RankSHAP: Shapley Value Based Feature Attributions for Learning to Rank (ICLR 2025 · similarity 0.85)
- Prospector Heads: Generalized Feature Attribution for Large Models & Data (ICML 2024 · similarity 0.84)
- Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model (NeurIPS 2025 · similarity 0.83)
- Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test for Causal Explanation (ICML 2024 · similarity 0.83)
- Data-faithful Feature Attribution: Mitigating Unobservable Confounders via Instrumental Variables (NeurIPS 2024 · similarity 0.83)
- One Wave To Explain Them All: A Unifying Perspective On Feature Attribution (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.