Research map / Label-efficient and robust learning / Recognition and long-tail
Feature Importance Disparities for Data Bias Investigations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes feature importance disparity (FID), an oracle-efficient algorithm that searches rich subgroups of protected attributes for large differences in a feature's importance (LIME, SHAP, saliency, linear coefficients) to aid data bias investigations.
Problem
Identifying structured subgroups of training data where a feature's influence on a classifier differs sharply from its population-level influence, to guide data bias investigations, since exhaustive search over rich subgroup classes is exponential.
Approach
Define feature importance disparity between a subgroup and the whole dataset for separable and linear importance notions, and find maximal-disparity rich subgroups via a Lagrangian game solved with exponentiated gradient and cost-sensitive classification oracle calls, with a linear-regression-based non-separable variant (LIN-FID) optimized by Adam over logistic-regression subgroup memberships.
Key results
- On Folktables with LIME, age is about 225 times more important on a found subgroup than on the whole population.
- Found subgroups show large out-of-sample AVG-FID across 16 dataset/notion pairs, e.g. COMPAS R SHAP age AVG-FID of 0.4 vs 0.06 for GerryFair subgroups.
- High AVG-FID subgroups show disparities in fairness metrics such as ECE and TPR, e.g. COMPAS R GRAD ECE difference 0.37.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Search rich subgroups rather than GerryFair subgroups | FID search · COMPAS R, Bank, and Folktables | AVG-FID: COMPAS R LIME: .04 -> .14; COMPAS R SHAP: .06 -> .4; COMPAS R GRAD: .03 -> .09; Bank LIME: .008 -> .03; Bank SHAP: .004 | Helped |
| Train a separate model for the identified Native-American and African-American male subgroup | COMPAS random forest · COMPAS two-year recidivism subgroup | false positive rate: drops the FPR in the subgroup by 7.5% | Helped |
| Train a separate model for the identified Native-American and African-American male subgroup | COMPAS random forest · COMPAS two-year recidivism subgroup | accuracy: increases accuracy by 7% | Helped |
| Remove priors-count | COMPAS random forest · COMPAS two-year recidivism | model performance | Hurt |
| Search rich subgroups instead of only marginal subgroups | FID search · Across datasets and importance notions | AVG-FID | Mixed |
| Search rich subgroups instead of only marginal subgroups | FID search · COMPAS R with GRAD, Arrested, No Charges feature | AVG-FID: 4 times less than on the full dataset | Helped |
Surprising findings
- Finding the maximal FID subgroup is NP-hard in the worst case (reduction from max-cut).
- Rich subgroups outperform marginal subgroups in only about half of cases.
Limitations
- Large FID does not necessarily imply unfairness; it is a diagnostic tool rather than proof of bias.
- Requires tabular data with predefined sensitive features.
- Feature importance notions are unstable and sometimes disagree.
Experimental setup
Datasets: Student COMPAS Bank Folktables
Benchmarks: Student COMPAS Bank Folktables
Metrics: AVG-FID FID demographic parity TPR FPR expected calibration error
Baselines: GerryFair marginal subgroups
Builds on: GerryFair LIME SHAP exponentiated gradient
Closest papers by meaning
- Subgroups Matter for Robust Bias Mitigation (ICML 2025 · similarity 0.86)
- Feature Importance Metrics in the Presence of Missing Data (ICML 2025 · similarity 0.85)
- DPA: A one-stop metric to measure bias amplification in classification datasets (NeurIPS 2025 · similarity 0.83)
- Improving Subgroup Robustness via Data Selection (NeurIPS 2024 · similarity 0.83)
- Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness (NeurIPS 2025 · similarity 0.83)
- The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.