Research map / Learning theory and algorithms / Fairness
Multiaccuracy and Multicalibration via Proxy Groups
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that proxy-sensitive attributes yield computable upper bounds on true multiaccuracy and multicalibration violations, and that post-processing with proxies reduces these worst-case violations.
Problem
Measuring and enforcing multiaccuracy and multicalibration fairness when sensitive group membership is missing or incomplete, where prior proxy methods only covered parity-based fairness notions.
Approach
Use learned proxy group functions with known misclassification error to derive provable upper bounds on true AE and ECE violations, and then post-process the model to be multiaccurate or multicalibrated with respect to the proxies so that the worst-case bounds shrink.
Key results
- On ACSIncome, the worst-case MC bound for the decision tree certifies ~0.21-multicalibration, reduced to ~0.13 after multicalibrating with respect to proxies (38% improvement).
- On ACSPubCov, the certified MC bound drops from ~0.25 to ~0.09 after proxy-based multicalibration.
- On CheXpert, the decision tree's worst-case violation is ~0.42 before adjustment and ~0.13 after; logistic regression and DenseNet-121 are certified at ~0.11 and ~0.12.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Multicalibrate the decision tree with respect to proxy groups | Decision tree · ACSIncome | worst-case multicalibration violation bound: 0.21 -> 0.13 | Helped |
| Multicalibrate the decision tree with respect to proxy groups | Decision tree · ACSPubCov | worst-case multicalibration violation bound: 0.25 -> 0.09 | Helped |
| Multicalibrate the decision tree with respect to proxy groups | Decision tree on DenseNet-121 features · CheXpert | worst-case multicalibration violation bound: 0.42 -> 0.13 | Helped |
| Adjust the logistic regression or end-to-end DenseNet-121 using proxy multicalibration | Logistic regression and end-to-end DenseNet-121 · CheXpert | worst-case multicalibration violation bound | No clear effect |
| Adjust already multiaccurate models using proxy groups | ACSIncome and ACSPubCov models · ACSIncome and ACSPubCov | multiaccuracy violation | No clear effect |
| Adjust already multiaccurate models using proxy groups | CheXpert models · CheXpert | multiaccuracy violation | No clear effect |
Surprising findings
- Logistic regression and random forest models on ACS were already highly multicalibrated, so proxy-based adjustment gave no benefit.
Limitations
- Bounds depend on proxy misclassification error and are upper bounds that may be loose; the paper notes tightness only for constructed distributions.
- Experiments are limited to two ACS tables and one chest X-ray dataset with a limited set of groups.
- Assumes the proxy developer can estimate proxy error err(g_hat) using true group labels.
Experimental setup
Datasets: ACSIncome ACSPublicCoverage CheXpert
Benchmarks: ACSIncome ACSPublicCoverage CheXpert
Metrics: multiaccuracy violation (AE) expected calibration error (ECE) worst-case violation bound mean squared error
Baselines: logistic regression decision tree random forest DenseNet-121
Builds on: multiaccuracy multicalibration proxy fairness auditing (Kallus et al.) multiaccuracy boosting
Closest papers by meaning
- Who’s the (Multi-)Fairest of Them All: Rethinking Interpolation-Based Data Augmentation Through the Lens of Multicalibration (AAAI 2025 · similarity 0.87)
- Fair Risk Control: A Generalized Framework for Calibrating Multi-group Fairness Risks (ICML 2024 · similarity 0.87)
- A Sequentially Fair Mechanism for Multiple Sensitive Attributes (AAAI 2024 · similarity 0.87)
- The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning (NeurIPS 2024 · similarity 0.87)
- Fairness without Harm: An Influence-Guided Active Sampling Approach (NeurIPS 2024 · similarity 0.87)
- On the Burden of Achieving Fairness in Conformal Prediction (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.