Research map / Learning theory and algorithms / Fairness
Preserving AUC Fairness in Learning with Noisy Protected Groups
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a distributionally robust AUC fairness method that bounds the total variation distance between clean and noisy protected-group distributions, estimating the noise ratio with CLIP and optimizing via SGDA with SAM.
Problem
AUC fairness methods assume clean protected-group labels; when group labels are noisy, group-level AUC gaps grow substantially, so fairness is not preserved in deployment.
Approach
Formulate AUC fairness as a DRO constraint over a TV-distance ball around the noisy-group pairwise distribution, with the TV bound estimated from the mismatch rate between noisy group labels and CLIP image-text similarity, and optimized via stochastic gradient descent-ascent with sharpness-aware perturbation.</1>
Key results
- On Adult at 0.5 noise, the method reaches fairness violation 0.0316 versus 0.0766 for AUCMax, 0.0374 for InterFairAUC and 0.0375 for MinimaxFairAUC.
- On FF++ with Xception, violation is 0.0090 versus 0.0111 for PG-FDD and 0.0288 for DAW-FDD, with AUC 0.9644 versus 0.9708 for PG-FDD.
- Ablation on EfficientNet-B4 shows the robust term reduces DFDC violation from 0.0191-level non-robust results to 0.0136 (full method) and improves Min/Max to 0.9771.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the robust AUC fairness method instead of MinimaxFairAUC | Tabular AUC fairness learning · Adult, noise 0.5 | Violation: 0.0375$\pm$0.0016 -> 0.0316$\pm$0.0002 | Helped |
| Use the robust AUC fairness method instead of InterFairAUC | Tabular AUC fairness learning · Bank, noise 0.5 | Violation: 0.0947$\pm$0.0017 -> 0.0876$\pm$0.0020 | Helped |
| Use the robust AUC fairness method instead of InterFairAUC | Tabular AUC fairness learning · Default, noise 0.5 | Violation: 0.0278$\pm$0.0026 -> 0.0243$\pm$0.0014 | Helped |
| Use the robust AUC fairness method instead of PG-FDD | Xception deepfake detector · FF++ | Violation: 0.0111 -> 0.0090 | Helped |
| Use the robust AUC fairness method instead of PG-FDD | Xception deepfake detector · DFDC | Violation: 0.0184 -> 0.0048 | Helped |
| Use the robust AUC fairness method instead of PG-FDD | Xception deepfake detector · DFD | Violation: 0.0113 -> 0.0069 | Helped |
| Use the robust AUC fairness method instead of PG-FDD | Xception deepfake detector · Celeb-DF | Violation: 0.1412 -> 0.0729 | Helped |
| Add SAM to the method (ablation) | EfficientNet-B4 deepfake detector · DFDC | Min/Max: 0.9656 -> 0.9771 | Helped |
| Add SAM to the method (ablation) | EfficientNet-B4 deepfake detector · DFD | Min/Max: 0.9721 -> 0.9760 | Helped |
| Add the robust DRO objective (ablation) | EfficientNet-B4 deepfake detector · DFDC | Violation: 0.0155 -> 0.0136 | Helped |
| Add the robust DRO objective (ablation) | Tabular AUC fairness learning · Adult, noise 0.1 | Violation: 0.0370$\pm$0.0006 -> 0.0332$\pm$0.0007 | Helped |
Surprising findings
- PG-FDD achieves higher overall AUC than the proposed method on deepfake benchmarks, so the method trades some utility for fairness.
- The Adult/Bank/Default robust-versus-non-robust comparison shows the non-robust variant is sometimes better on specific metrics (e.g., Bank violation at 0.2 noise).
Limitations
- No concrete noise-estimation strategy for tabular data; CLIP-based estimation is only instantiated for images.
- Ensuring utility and fairness simultaneously under noisy groups remains open; the method has lower AUC than some baselines on deepfake datasets.
Experimental setup
Datasets: Adult Bank Default FF++ DFDC DFD Celeb-DF
Benchmarks: Adult Bank Default FF++ DFDC DFD Celeb-DF
Metrics: AUC AUC fairness violation Min/Max AUC ratio
Baselines: AUCMax InterFairAUC MinimaxFairAUC DAG-FDD DAW-FDD PG-FDD Original
Builds on: MinimaxFairAUC distributionally robust optimization sharpness-aware minimization DAW-FDD CLIP
Closest papers by meaning
- Minimax AUC Fairness: Efficient Algorithm with Provable Convergence (AAAI 2023 · similarity 0.91)
- Optimal Fair Learning Robust to Adversarial Distribution Shift (ICML 2025 · similarity 0.86)
- FROC: Building Fair ROC from a Trained Classifier (AAAI 2025 · similarity 0.86)
- Multiaccuracy and Multicalibration via Proxy Groups (ICML 2025 · similarity 0.84)
- OxonFair: A Flexible Toolkit for Algorithmic Fairness (NeurIPS 2024 · similarity 0.84)
- Fairness without Harm: An Influence-Guided Active Sampling Approach (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.