Research map / Vision-language and multimodal / Human and embodied modeling
Subgroups Matter for Robust Bias Mitigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that the choice of subgroups used for bias mitigation strongly determines whether gDRO, resampling, DomainInd and CFair help or hurt, and explains this via the KL divergence between subgroup-reweighted training and unbiased test distributions.
Problem
Bias mitigation methods inconsistently beat ERM, and the subgroup definition step used to drive mitigation is rarely studied; it is unclear when and why these methods fail under spurious correlations.
Approach
Systematically varies subgroup definitions (coarse, fine, intersectional, random, noisy) across four bias mitigation methods and four semi-synthetic vision and language datasets, and proposes that a grouping is effective when reweighting it can recover the unbiased test distribution (minimum KL divergence), so fairness for one attribute can be best improved by mitigating with a different grouping.
Key results
- Subgroup choice shifts AUC relative to ERM by more than 0.07 in either direction; some groupings reduce performance by up to 0.08.
- Minimum KL divergence achievable by reweighting subgroups correlates with unbiased AUC at 0.73 to 0.97 across datasets for resampling and gDRO.
- Using (A,Y) subgroups gave min accuracy across S up to 0.21 higher and accuracy gaps up to 10 times smaller than using S-based groupings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AY groups rather than S groups for gDRO or resampling (ablation) | Bias mitigation on the semi-synthetic datasets · Unbiased test set | AUC | Helped |
| Use S groups for mitigation (ablation) | gDRO and resampling · Unbiased test set across four datasets | AUC: 10 out of 16 experiments | Hurt |
| Use random subgroups for mitigation (ablation) | Bias mitigation methods · Unbiased test set across four datasets | AUC | Hurt |
| Increase granularity from AY to AY_8 or SY to SY_8 (ablation) | gDRO and resampling · Unbiased test set | AUC | No clear effect |
| Use coarse rather than granular real subgroup labels (ablation) | gDRO and resampling on Civil_comments · Civil_comments unbiased test | AUC | No clear effect |
| Add noise to AY subgroup annotations above 10% (ablation) | gDRO and resampling · Unbiased test set | AUC: 10% | Hurt |
| Use AY groups rather than S groups for mitigation (ablation) | gDRO and resampling · S subgroups, averaged across experiments | minimum accuracy across S groups: 0.705 ± 0.025 -> 0.779 ± 0.02 | Helped |
| Use AY groups rather than S groups for mitigation (ablation) | gDRO and resampling · S subgroups, averaged across experiments | accuracy gap across S groups: 0.064 ± 0.045 -> 0.032 ± 0.03 | Helped |
| Use YSA intersectional groups rather than S groups for mitigation (ablation) | gDRO and resampling · S subgroups, averaged across experiments | minimum accuracy across S groups: 0.705 ± 0.025 -> 0.792 ± 0.013 | Helped |
| Use YSA intersectional groups rather than S groups for mitigation (ablation) | gDRO and resampling · S subgroups, averaged across experiments | accuracy gap across S groups: 0.064 ± 0.045 -> 0.02 ± 0.022 | Helped |
| Use AY validation groups for JTT model and hyperparameter selection (ablation) | JTT · MNIST unbiased test | AUC: 0.792 ± 0.057 -> 0.925 ± 0.019 | Helped |
| Use AY validation groups for JTT model and hyperparameter selection (ablation) | JTT · Civil_comments unbiased test | AUC: 0.805 ± 0.017 -> 0.831 ± 0.004 | Helped |
Surprising findings
- Some subgroupings leave the method worse than no mitigation at all, e.g. S subgroups harmed performance in 10 of 16 experiments.
- Improving fairness with respect to S is best achieved by mitigating with (A,Y) subgroups rather than S subgroups.
Limitations
- Only studies spurious correlation bias in a semi-synthetic, controlled setting with small datasets.
- KL analysis assumes divergence is fully attributable to P(Y,S,A) and requires full knowledge of test-time distribution shift.
Experimental setup
Datasets: MNIST CheXpert CelebA Civil_comments
Benchmarks: MNIST CheXpert CelebA Civil_comments
Metrics: AUC worst-group accuracy accuracy gap KL divergence to unbiased distribution
Baselines: ERM gDRO resampling DomainInd CFair Just Train Twice
Builds on: group DRO Just Train Twice CFair DomainInd
Closest papers by meaning
- Discover and Mitigate Multiple Biased Subgroups in Image Classifiers (CVPR 2024 · similarity 0.92)
- Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness (NeurIPS 2025 · similarity 0.87)
- Improving Subgroup Robustness via Data Selection (NeurIPS 2024 · similarity 0.87)
- The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations (NeurIPS 2024 · similarity 0.87)
- Mitigating Label Bias in Machine Learning: Fairness through Confident Learning (AAAI 2024 · similarity 0.86)
- FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.