Research map / Label-efficient and robust learning / Recognition and long-tail
On Feature Learning in the Presence of Spurious Correlations
Pavel Izmailov, Polina Kirichenko, Nate Gruver, Andrew Gordon Wilson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply DFR (retrain last layer on held-out balanced data) to ERM features | ERM, ResNet-50 · Waterbirds | WGA (%): 68.9 -> 91.1 | Improved |
| use group DRO / RWG / RWY training instead of ERM before applying DFR | feature extractor · Waterbirds, CelebA, FMOW, text | DFR WGA: ERM+DFR 91.1 vs GDRO-ES+DFR 89.9 on Waterbirds | No clear effect |
| early stopping for group-robust methods before DFR (ablation) | group DRO, RWG, RWY · Waterbirds, CelebA | DFR WGA: GDRO 88.2 vs GDRO-ES 89.9 (Waterbirds) | No clear effect |
| use stronger pretrained architecture (ConvNext Large IN22k) instead of ResNet-50 (ablation) | ERM + DFR · Waterbirds | DFR WGA: 91.1 -> 97.2 | Improved |
| stronger base model on MultiNLI (DeBERTa-Large vs BERT-Base) (ablation) | ERM + DFR · MultiNLI | DFR WGA: 73.8 -> 84.8 | Improved |
| finetune feature extractor on Waterbirds vs only pretrained features (ablation) | ResNet-50 IN1k + DFR · Waterbirds | DFR WGA: 88.2 -> 92.9 | Improved |
| finetune ConvNext Large IN22k on Waterbirds (ablation) | ConvNext Large + DFR · Waterbirds | DFR WGA: 94 -> 94.3 | No clear effect |
| finetune feature extractor on target data (ablation) | models on CelebA/FMOW · CelebA | DFR WGA: 88.3 -> 92.2 | Improved |
| use supervised ImageNet pretraining vs random init (ablation) | ResNet-50 · Waterbirds, FMOW, CelebA | DFR WGA | Improved |
| add weight decay (non-zero) before DFR (ablation) | ResNet-50 IN1k · CelebA | base WGA vs DFR WGA: base WGA +~5%; DFR WGA best at zero weight decay | Mixed |
| MixUp augmentation (ablation) | ResNet-50 · FMOW | DFR WGA: 42% -> 30% | Hurt |
| Random Erasing augmentation (ablation) | ResNet-50 · Waterbirds | DFR WGA | Hurt |
Closest papers by meaning
- Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples (CVPR 2025 · similarity 0.89)
- Complexity Matters: Feature Learning in the Presence of Spurious Correlations (ICML 2024 · similarity 0.88)
- Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (ICLR 2025 · similarity 0.87)
- Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations (CVPR 2024 · similarity 0.86)
- How Spurious Features are Memorized: Precise Analysis for Random and NTK Features (ICML 2024 · similarity 0.86)
- Label-Efficient Group Robustness via Out-of-Distribution Concept Curation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
