Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Prompting is a Double-Edged Sword: Improving Worst-Group Robustness of Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows foundation models are poor zero-shot at worst-group label prediction under spurious correlations but accurate at predicting the spurious attribute, and proposes PfR, which zero-shot labels confounders then runs group DRO over predicted groups, nearly matching oracle group DRO.
Problem
Foundation models (zero-shot and few-shot) have poor worst-group robustness under spurious correlations in downstream data, and scaling pretraining does not fix this; group annotations for confounders are typically unavailable.
Approach
Use a zero-shot foundation model prompted with a text description of the spurious attribute to predict group annotations on labeled data, then train a linear head with group DRO over the predicted (label, confounder) groups, plus a theoretical analysis of spectral contrastive pretraining explaining why zero-shot label prediction fails while confounder prediction succeeds.
Key results
- PfR reaches 91.05 worst-group accuracy on Waterbirds vs 59.38 for zero-shot and 70.71 for ERM, close to oracle Group DRO at 93.23.
- Averaged across datasets, PfR reduces worst-group error by 47% relative to zero-shot and 52% relative to ERM.
- Zero-shot prediction of the spurious attribute alone reaches about 95% average accuracy across standard spurious correlation benchmarks, while label prediction with confounder description in prompt does not improve worst-group accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PfR instead of the strongest reported baseline | Waterbirds classifiers · Waterbirds | worst-group accuracy: 90.13 -> 91.05 | Helped |
| use PfR instead of the strongest reported baseline | CelebA classifiers · CelebA | worst-group accuracy: 88.12 -> 88.05 | No clear effect |
| use PfR instead of the strongest reported baseline | CivilComments classifiers · CivilComments | worst-group accuracy: 72.73 -> 77.83 | Helped |
| use PfR instead of the strongest reported baseline | MNLI classifiers · MNLI | worst-group accuracy: 81.54 -> 81.28 | No clear effect |
| use PfR instead of the strongest reported baseline | CXR-Drain classifiers · CXR-Drain | worst-group accuracy: 59.37 -> 68.55 | Helped |
| add the confounder description to the label prompt (ablation) | zero-shot classifiers · Waterbirds | worst-group accuracy: 59.38 -> 57.38 | Hurt |
| add the confounder description to the label prompt (ablation) | zero-shot classifiers · CelebA | worst-group accuracy: 77.69 -> 78.54 | No clear effect |
| add the confounder description to the label prompt (ablation) | zero-shot classifiers · CivilComments | worst-group accuracy: 59.25 -> 54.29 | Hurt |
| add the confounder description to the label prompt (ablation) | zero-shot classifiers · MNLI | worst-group accuracy: 76.54 -> 75.73 | No clear effect |
| instruct the model to ignore the confounder (ablation) | zero-shot classifiers · CivilComments | worst-group accuracy: 59.25 -> 52.81 | Hurt |
| use in-context learning with 128 examples instead of zero-shot inference (ablation) | language task classifiers · CivilComments and MNLI | worst-group accuracy | Mixed |
Surprising findings
- Adding the spurious attribute description to the zero-shot label prompt does not improve worst-group accuracy, and in-context learning with 128 examples does not improve robustness gap and can hurt it on MNLI.
- Scaling pretraining data and model size does not improve zero-shot worst-group label accuracy, even though it reduces the confounder-prediction gap.
Limitations
- Requires the user to know and describe the spurious attribute in advance; automatic discovery of spurious attributes is left to future work.
- Still needs a labeled downstream dataset and post-training intervention; zero-shot robustness is not achieved.
- Theoretical analysis is limited to a simplified linear setup with spectral contrastive loss.
Experimental setup
Datasets: Waterbirds CelebA CivilComments MNLI Chest X-ray 14
Benchmarks: Waterbirds CelebA CivilComments MNLI CXR-Drain
Metrics: worst-group accuracy average accuracy
Baselines: Zero-shot ERM JTT Group DRO DebiAN EIIL ReBias In-context learning
Builds on: Group DRO CLIP JTT spectral contrastive loss
Closest papers by meaning
- Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models (ICLR 2024 · similarity 0.87)
- Label-Efficient Group Robustness via Out-of-Distribution Concept Curation (CVPR 2024 · similarity 0.86)
- LR0.FM: Low-Res Benchmark and Improving robustness for Zero-Shot Classification in Foundation Models (ICLR 2025 · similarity 0.85)
- Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition (ICLR 2025 · similarity 0.85)
- Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples (CVPR 2025 · similarity 0.85)
- Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.