Research map / Learning theory and algorithms / Fairness
Counterfactual Fairness by Combining Factual and Counterfactual Predictions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves that combining factual and counterfactual predictions from a Bayes-optimal (possibly unfair) predictor gives optimal counterfactually fair predictors, quantifies the inherent CF-accuracy trade-off, and proposes PCF/CRM methods that remain robust under imperfect counterfactual estimation.
Problem
Counterfactual fairness often degrades predictive performance, but the optimal accuracy achievable under perfect counterfactual fairness and its trade-off with predictive performance were unknown, especially under imperfect causal knowledge.
Approach
Under invertible causal models, the Bayes-optimal counterfactually fair predictor is the average of the unconstrained optimal predictions at the factual point and its counterfactual; the excess risk of this constraint is characterized exactly (squared loss: sigma_A^2 times a U-level dependence between Y and A; cross-entropy: I(A;Y|U)). A plug-in method (PCF) mixes outputs of any pretrained predict
Key results
- On synthetic data with ground-truth counterfactuals, PCF achieves perfect CF (TE=0) with the lowest error among CF methods (CFU, CFR, ECOCF), matching the derived optimality.
- Under counterfactual estimation error, PCF degrades more than PCF-Analytic, and PCF-CRM improves CF and error over CFU and CFR on the semi-synthetic Law School dataset.
- The paper proves the excess risk of perfect CF equals sigma_A^2 * E_U[(E[Y|U,A=a]-E[Y|U,A=1-a])^2] for squared loss and I(A;Y|U) for cross-entropy loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine factual and counterfactual predictions with PCF | KNN predictors · Linear-Reg, Cubic-Reg, Linear-Cls, and Cubic-Cls with ground-truth counterfactuals | Error | Helped |
| Use PCF with ground-truth counterfactuals | KNN predictors · Linear-Reg, Cubic-Reg, Linear-Cls, and Cubic-Cls with ground-truth counterfactuals | TE | No clear effect |
| Use PCF rather than CFE or CFR as counterfactual estimation noise increases | Synthetic datasets · Linear-Reg, Cubic-Reg, Linear-Cls, and Cubic-Cls under controllable estimation error | Error | Helped |
| Use PCF rather than CFE or CFR as counterfactual estimation noise increases | Synthetic datasets · Linear-Reg, Cubic-Reg, Linear-Cls, and Cubic-Cls under controllable estimation error | TE | Helped |
| Increase counterfactual estimation error | CFE, CFR, and PCF · Synthetic datasets under controllable estimation error | Error | Hurt |
| Increase counterfactual estimation error | CFE, CFR, and PCF · Synthetic datasets under controllable estimation error | TE | Hurt |
| Use the analytic Bayes-optimal predictor instead of an ERM-trained predictor in PCF | PCF · Synthetic datasets under counterfactual estimation error | Error | Helped |
| Use PCF-CRM rather than CFU or CFR | Law School Success semi-synthetic dataset · Law School Success | Error | Helped |
| Use PCF-CRM rather than CFU or CFR | Law School Success semi-synthetic dataset · Law School Success | TE | Helped |
| Use ERM rather than fairness-oriented methods | Law School Success semi-synthetic dataset · Law School Success | Error | Helped |
| Use ERM rather than fairness-oriented methods | Law School Success semi-synthetic dataset · Law School Success | TE | Hurt |
| Mix predictors to optimize the trade-off between counterfactual fairness and error | PCF-CRM · Law School Success semi-synthetic dataset | Error | Helped |
Surprising findings
- PCF-Analytic (Bayes-optimal predictor) is more robust to counterfactual estimation error than PCF using an ERM-trained predictor, which the authors attribute to poor ERM training on the estimated counterfactual distribution.
Limitations
- Requires invertibility between X and U given A, which may be restrictive in practice.
- Experiments use ground-truth or simulated counterfactuals; real counterfactuals are hard to obtain.
- Theory assumes access to a Bayes-optimal predictor, which practical ERM models only approximate.
Experimental setup
Datasets: Linear-Reg Cubic-Reg Linear-Cls Cubic-Cls Law School Success
Benchmarks: Linear-Reg Cubic-Reg Linear-Cls Cubic-Cls Law School Success (semi-synthetic)
Metrics: Error Total Effect (TE) TE_0 TE_1
Baselines: ERM CFU CFR ECOCF PCF-Analytic
Builds on: counterfactual fairness structural causal models equal counterfactual opportunity counterfactual fairness with unobserved confounders (CFU)
Closest papers by meaning
- FairPFN: A Tabular Foundation Model for Causal Fairness (ICML 2025 · similarity 0.88)
- Reconciling Predictive and Statistical Parity: A Causal Approach (AAAI 2024 · similarity 0.88)
- Fairness on Principal Stratum: A New Perspective on Counterfactual Fairness (ICML 2025 · similarity 0.88)
- Causal Fairness under Unobserved Confounding: A Neural Sensitivity Framework (ICLR 2024 · similarity 0.88)
- Towards counterfactual fairness through auxiliary variables (ICLR 2025 · similarity 0.87)
- Fairness-Accuracy Trade-Offs: A Causal Perspective (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.