Research map / Label-efficient and robust learning / Recognition and long-tail
Unsupervised Concept Discovery Mitigates Spurious Correlations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoBalT, which discovers discrete concepts from slot-attention object-centric representations (vector-quantized, teacher-student) and uses concept-aware importance sampling to train a robust classifier without group annotations.
Problem
Spurious correlations cause models to fail on minority subgroups, but group annotations needed to mitigate them are costly or unavailable.
Approach
Use unsupervised object-centric slot representations, vector-quantized into a learned concept dictionary, as data-driven proxies for subgroups, then resample training data so that rare concepts and classes within concepts are balanced.
Key results
- On Waterbirds, CoBalT-ig reaches 89.0% worst-group accuracy without group labels, about 3% above ULA (86.1) and XRM (86.1), per the paper's claim.
- On UrbanCars with BG+CoObj correlations, CoBalT-ig achieves 80.0% worst-group accuracy versus 76.9% for SPARE.
- On CelebA with validation-group-free early stopping, CoBalT-ig reaches 89.2% worst-group accuracy versus 78.0% for MaskTune.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use concept discovery and concept-aware sampling | CoBalT · Waterbirds | worst-group accuracy: 86.1 -> 89.0 | Helped |
| use concept discovery and concept-aware sampling | CoBalT · Urban Cars (BG+CoObj) | worst-group accuracy: 76.9±1.8 -> 80.0±2.8 | Helped |
| use concept discovery and concept-aware sampling | CoBalT · IN-9L Mixed-same | accuracy: 91.1 -> 91.2 | Helped |
| use concept discovery and concept-aware sampling | CoBalT · IN-9L Mixed-rand | accuracy: 78.6 -> 80.1 | Helped |
| use concept discovery and concept-aware sampling | CoBalT · IN-9L Only-FG | accuracy: 88.1 -> 90.0 | Helped |
| use concept discovery and concept-aware sampling | CoBalT · CelebA, no validation groups | worst-group accuracy: 78.0±1.2 -> 89.2±1.2 | Helped |
| vary the number of slots; over-segment with more slots (ablation) | CoBalT · Waterbirds | worst-group accuracy | Hurt |
| use codebook size 8 instead of 4 (ablation) | CoBalT · CelebA, 4 slots | worst-group accuracy: 74.4 -> 88.3 | Helped |
| use codebook size 8 instead of 4 (ablation) | CoBalT · CelebA, 4 slots | average accuracy: 93.5 -> 92.6 | Hurt |
| use codebook size 12 instead of 4 (ablation) | CoBalT · CelebA, 4 slots | worst-group accuracy: 74.4 -> 86.7 | Helped |
| use codebook size 12 instead of 4 (ablation) | CoBalT · CelebA, 4 slots | average accuracy: 93.5 -> 89.7 | Hurt |
| use codebook size 16 instead of 4 (ablation) | CoBalT · CelebA, 4 slots | worst-group accuracy: 74.4 -> 88.3 | Helped |
Surprising findings
- Performance degrades on CMNIST and CelebA, where concepts are less spatially separable, and the authors attribute this to the spatial-separability assumption of slot-based decomposition.
Limitations
- Assumes concepts are spatially separable, limiting gains on CMNIST and CelebA.
- Results depend on hyperparameters such as slot count and codebook size; over-segmentation degrades worst-group accuracy.
- Evaluation restricted to vision datasets.
Experimental setup
Datasets: CMNIST CelebA Waterbirds UrbanCars Background Challenge ImageNet-9 BAR
Benchmarks: CMNIST Waterbirds CelebA UrbanCars ImageNet-9 (IN-9L) BAR
Metrics: worst group accuracy average accuracy
Baselines: GDRO DFR AFR ULA MaskTune XRM SPARE JTT CnC GEORGE LfF CIM
Builds on: slot attention DINO vector quantization JTT SPARE
Closest papers by meaning
- Label-Efficient Group Robustness via Out-of-Distribution Concept Curation (CVPR 2024 · similarity 0.85)
- Prior-Constrained Association Learning for Fine-Grained Generalized Category Discovery (AAAI 2025 · similarity 0.84)
- Nonparametric Identification of Latent Concepts (ICML 2025 · similarity 0.84)
- Concept-Based Unsupervised Domain Adaptation (ICML 2025 · similarity 0.84)
- Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions (NeurIPS 2025 · similarity 0.84)
- Unleashing the Potential of Model Bias for Generalized Category Discovery (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.