Research map / Label-efficient and robust learning / Recognition and long-tail
Label-Efficient Group Robustness via Out-of-Distribution Concept Curation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Concept Correction, which labels training samples' spurious attributes via CAV similarity to small curated (often generated) concept image sets, and instantiates it as Concept DRO to reach worst-group accuracy near label-based methods without group labels.
Problem
Worst-group accuracy under spurious correlations typically requires group labels on training or validation data, which are expensive to obtain; existing label-free methods such as GEORGE struggle on complex real-world features.
Approach
Represent a spurious attribute as a small set of out-of-distribution concept images, train a concept activation vector (CAV) on ERM features to score each training and validation sample, cluster scores with a GMM to infer spurious labels, and then run Group DRO with these inferred group labels (and surrogate worst-group validation).
Key results
- CDRO outperforms JTT by 5% on Waterbirds and 6.5% on CelebA in worst-group accuracy among methods without group labels, and beats GEORGE with 88.8% vs 76.2% worst-group on Waterbirds and 88.0% vs 54.9% on CelebA.
- Pseudo-group labels inferred by CDRO reach precision/recall of 87.0/96.9 on CMNIST, 76.3/83.9 on Waterbirds and 78.0/78.5 on CelebA.
- Even with only 4 concept images CDRO beats GEORGE on Waterbirds and CelebA; with distant prompts CelebA worst-group accuracy drops to 75.6% and Waterbirds to 53.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CDRO instead of ERM | Image classification · CMNIST | worst-group accuracy (%): 0.0 (0.0) -> 69.3 (2.7) | Helped |
| Use CDRO instead of GEORGE | Image classification · Waterbirds | worst-group accuracy (%): 76.2 (2.0) -> 88.8 (0.2) | Helped |
| Use CDRO instead of GEORGE | Image classification · CelebA | worst-group accuracy (%): 54.9 (1.9) -> 88.0 (0.7) | Helped |
| Use CDRO instead of JTT | Image classification · Waterbirds | worst-group accuracy (%): 83.8 (1.2) -> 88.8 (0.2) | Helped |
| Use CDRO instead of JTT | Image classification · CelebA | worst-group accuracy (%): 81.5 (1.7) -> 88.0 (0.7) | Helped |
| Use CDRO instead of GEORGE | Image classification · CMNIST | worst-group accuracy (%): 76.4 (2.3) -> 69.3 (2.7) | Hurt |
| Use CDRO instead of CNC | Image classification · Waterbirds | worst-group accuracy (%): 88.5 (0.3) -> 88.8 (0.2) | No clear effect |
| Use CDRO instead of CNC | Image classification · CelebA | worst-group accuracy (%): 88.8 (0.9) -> 88.0 (0.7) | No clear effect |
| Use somewhat near concepts instead of near concepts (ablation) | CDRO · Waterbirds | worst-group accuracy (%): 88.5 -> 80.1 | Hurt |
| Use distant concepts instead of near concepts (ablation) | CDRO · Waterbirds | worst-group accuracy (%): 88.5 -> 53.1 | Hurt |
| Use somewhat near concepts instead of near concepts (ablation) | CDRO · CelebA | worst-group accuracy (%): 87.9 -> 87.2 | No clear effect |
| Use distant concepts instead of near concepts (ablation) | CDRO · CelebA | worst-group accuracy (%): 87.9 -> 75.6 | Hurt |
Surprising findings
- Concept sets generated with only two-line Stable Diffusion prompts achieve performance competitive with methods using tens of thousands of labeled samples.
- Concept sets from very general prompts (distant distribution) substantially degrade Waterbirds worst-group accuracy to 53.1%.
Limitations
- Performance depends on concept set quality; distant or off-target prompts markedly reduce worst-group accuracy.
- Spurious attribute must be representable as a concept; CMNIST uses synthetic colored digits.
- Lags label-based methods such as Group DRO on Waterbirds and CelebA (e.g., 88.8 vs 89.9 worst-group on Waterbirds).
Experimental setup
Datasets: CMNIST Waterbirds CelebA MNIST
Benchmarks: CMNIST Waterbirds CelebA
Metrics: worst-group accuracy mean accuracy precision recall
Baselines: Group DRO JTT CNC SSA AFR GEORGE ERM
Builds on: Group DRO Concept Activation Vectors (TCAV) GEORGE Just Train Twice Correct-n-Contrast
Closest papers by meaning
- Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples (CVPR 2025 · similarity 0.88)
- Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference (ICML 2024 · similarity 0.87)
- Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness (CVPR 2025 · similarity 0.87)
- Prompting is a Double-Edged Sword: Improving Worst-Group Robustness of Foundation Models (ICML 2024 · similarity 0.86)
- ExMap: Leveraging Explainability Heatmaps for Unsupervised Group Robustness to Spurious Correlations (CVPR 2024 · similarity 0.86)
- The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.