Research map / Label-efficient and robust learning / Recognition and long-tail
Mitigating Spurious Correlations via Disagreement Probability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPR, a two-stage resampling debiasing method that upsamples bias-conflicting samples using the disagreement probability of a GCE-trained biased model, without bias labels.
Problem
Spurious correlations cause ERM models to fail on bias-conflicting groups, and debiasing typically needs bias labels that are costly or unavailable; the optimal weighting of bias-conflicting samples is unclear.
Approach
Derives from a min-max group-loss objective a weighted loss whose weights are proportional to the bias-conflicting probability; approximates this probability by the disagreement 1 - p_bias(y|x) of a GCE-trained biased model, then resamples with these weights and initializes the debiased model from the biased model.
Key results
- On BFFHQ, DPR reaches 76.80% bias-conflicting accuracy, 6.20% above LC, and 87.57% unbiased accuracy.
- On MB-MNIST with 30% bias-conflicting ratio, DPR reaches 94.04% unbiased accuracy, 3.28% above PGD.
- Ablation on BFFHQ shows GCE and augmentation together raise unbiased and conflicting accuracy by 9.07% and 19.33% over the initialization-only setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DPR instead of PGD | C-MNIST · C-MNIST 0.5% | accuracy: 96.88 (0.28) -> 97.52 (0.33) | Helped |
| Use DPR instead of PGD | MB-MNIST · MB-MNIST 30% | accuracy: 90.76 (1.84) -> 94.04 (0.26) | Helped |
| Use DPR instead of PGD | BFFHQ · BFFHQ unbiased test | accuracy: 84.20 (1.15) -> 87.57 (1.22) | Helped |
| Use DPR instead of LC | BFFHQ · BFFHQ bias-conflicting test | accuracy: 70.60 (0.60) -> 76.80 (2.51) | Helped |
| Add model initialization, GCE, and data augmentation (ablation) | DPR · BFFHQ unbiased test | accuracy: 77.07 (1.16) -> 87.57 (1.22) | Helped |
| Add model initialization, GCE, and data augmentation (ablation) | DPR · BFFHQ bias-conflicting test | accuracy: 54.60 (2.43) -> 76.80 (2.51) | Helped |
| Add GCE and augmentation without initialization (ablation) | DPR · C-MNIST 0.5% unbiased test | accuracy: 66.13 (0.51) -> 66.47 (1.74) | Helped |
| Add GCE and augmentation without initialization (ablation) | DPR · C-MNIST 5% unbiased test | accuracy: 95.32 (0.42) -> 95.21 (0.14) | Hurt |
| Add GCE while using initialization (ablation) | DPR · C-MNIST 0.5% unbiased test | accuracy: 89.06 (0.62) -> 95.94 (0.45) | Helped |
| Add data augmentation to initialization plus GCE (ablation) | DPR · BFFHQ bias-conflicting test | accuracy: 63.40 (3.67) -> 76.80 (2.51) | Helped |
| Increase GCE q from 0.3 to 0.9 (ablation) | DPR · C-MNIST 0.5% unbiased test | accuracy: 95.53 (0.74) -> 97.66 (0.06) | Helped |
| Increase GCE q from 0.3 to 0.9 (ablation) | DPR · MB-MNIST 30% unbiased test | accuracy: 95.32 (0.71) -> 93.77 (0.19) | Hurt |
Surprising findings
- Randomly initialized and ImageNet-pretrained models show similar losses on bias-aligned and bias-conflicting groups, so initialization from the biased model is needed to satisfy the assumption.
- Resampling consistently outperforms reweighting for the derived weighted loss on C-MNIST.
Limitations
- Effectiveness depends on how well the biased model captures the spurious correlation structure.
- Two-stage approach complicates training and adds hyperparameters (temperature tau, GCE q).
- Results for several baselines are copied from Ahn et al. rather than rerun.
Experimental setup
Datasets: Colored MNIST Multi-bias MNIST Biased action recognition (BAR) Biased FFHQ (BFFHQ) CelebA CivilComments-WILDS
Benchmarks: Colored MNIST Multi-bias MNIST BAR BFFHQ CelebA CivilComments-WILDS
Metrics: unbiased test accuracy bias-conflicting accuracy average accuracy worst-group accuracy
Baselines: ERM JTT DFA CNC PGD LC
Builds on: Group DRO LfF JTT LISA GCE loss
Closest papers by meaning
- Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples (CVPR 2025 · similarity 0.88)
- SEBRA : Debiasing through Self-Guided Bias Ranking (ICLR 2025 · similarity 0.87)
- Fairness under Covariate Shift: Improving Fairness-Accuracy Tradeoff with Few Unlabeled Test Samples (AAAI 2024 · similarity 0.86)
- A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective (NeurIPS 2024 · similarity 0.86)
- An Effective Theory of Bias Amplification (ICLR 2025 · similarity 0.85)
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.