Research map / Learning theory and algorithms / Fairness
The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts graphical causal sensitivity analysis (autobounds) to fairness metrics under proxy label, selection, and extra-classificatory policy biases in the oblivious setting, showing common parity metrics are fragile to minor biases across 14 datasets.
Problem
Fairness metric evaluations are assumed reliable, but real-world measurement biases (proxy labels, selection, policies) can make parity assessments uninformative, especially when the covariates are unobserved.
Approach
Map measurement biases to discrete causal DAGs with latent projection over the unobserved covariates, then bound fairness metrics via the autobounds polynomial-programming framework over a sensitivity parameter, yielding worst-case bounds on parity metrics.
Key results
- Automatically derived bounds matched algebraically derived bounds for COMPAS proxy-bias plots under several assumption variants.
- Combining proxy (5%) and selection (5%) bias on Adult drove the upper bound of equalized odds to its maximum possible value.
- Across 14 datasets, demographic parity was the most robust metric while predictive parity metrics were the most fragile to biased outcomes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Assess demographic parity instead of more outcome-dependent parity metrics under measurement bias | Parity metric evaluation · 14 FairML benchmark datasets | sensitivity of parity bounds | Helped |
| Assess equalized odds instead of FPR or FNR parity under measurement bias | Parity metric evaluation · 14 FairML benchmark datasets | sensitivity of parity bounds | Hurt |
| Use predictive parity metrics rather than outcome-conditioned metrics | Parity metric evaluation · 14 FairML benchmark datasets | sensitivity to biased outcomes | Hurt |
| Combine proxy-label bias and selection bias | Equalized-odds predictor · Adult | equalized odds upper bound: 5% proxy labels and 5% of the population being unmeasured | Hurt |
| Remove the causal edge from X to the proxy label Y_P (ablation) | COMPAS proxy-label sensitivity analysis · COMPAS | FNR bound informativeness | Helped |
| Remove the causal edge from X to the proxy label Y_P (ablation) | COMPAS proxy-label sensitivity analysis · COMPAS | FPR and PPV bound informativeness | Helped |
| Assume P(Y_P=0 | Y=1)=0 instead of P(Y_P=1 | Y=0)=0 (ablation) | COMPAS proxy-label sensitivity analysis · COMPAS | sensitivity bounds | Mixed |
| Combine proxy-label and selection bias | Parity metric evaluation · Adult | FPR and FNR parity bound range | Mixed |
| Combine proxy-label and selection bias | Parity metric evaluation · Adult | PPP and NPP parity bound range | Mixed |
Surprising findings
- Even small amounts of measurement bias can render parity evaluations meaningless, with bounds reaching the maximum possible value.
- Dropping a single edge (X to Y_P) under a proxy assumption fully identifies FNR and tightens FPR and PPV bounds, showing sensitivity analysis can reveal new identification results.
Limitations
- Restricted to discrete variables and the oblivious setting without covariates X.
- Causal graphs are context-specific and assumed plausible rather than validated; unrealistic assumptions can give false security.
- Does not cover interference and other biases that are hard to express graphically.
Experimental setup
Datasets: Adult COMPAS recidivism COMPAS violent recidivism Dutch census German credit Credit card clients Bank marketing KDD Census-Income Communities and Crime Diabetes Ricci Student-Mathematics
Benchmarks: COMPAS Adult Dutch census 14 canonical fairness datasets
Metrics: FPR parity FNR parity PPV parity NPV parity equalized odds demographic parity upper and lower sensitivity bounds
Baselines: algebraically derived bounds (proxy bias, Fogliato et al.-style analysis) reductions approach (Agarwal et al.)
Builds on: autobounds latent projection reductions approach for fair classification
Closest papers by meaning
- Reconciling Predictive and Statistical Parity: A Causal Approach (AAAI 2024 · similarity 0.92)
- Causal Fairness under Unobserved Confounding: A Neural Sensitivity Framework (ICLR 2024 · similarity 0.90)
- A Sequentially Fair Mechanism for Multiple Sensitive Attributes (AAAI 2024 · similarity 0.89)
- FairPFN: A Tabular Foundation Model for Causal Fairness (ICML 2025 · similarity 0.89)
- Interventional Fairness on Partially Known Causal Graphs: A Constrained Optimization Approach (ICLR 2024 · similarity 0.89)
- Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.