Research map / Vision-language and multimodal / Human and embodied modeling
Discover and Mitigate Multiple Biased Subgroups in Image Classifiers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DIM, which uses PLS decomposition of CLIP image features supervised by classifier training dynamics to discover, interpret (via CLIP retrieval and ChatGPT summaries), and mitigate multiple unknown biased subgroups in image classifiers.
Problem
Discovering and mitigating multiple unknown biased subgroups in image classifiers when subgroup labels are unavailable, a harder and more practical setting than single-bias discovery.
Approach
Integrates the classifier's training dynamics (correctness, logit, loss) as supervision into Partial Least Squares decomposition of CLIP image embeddings to extract multiple subgroup directions, then interprets them with CLIP retrieval plus LLM summarization and mitigates bias via data-centric filtering and model-centric pseudo-label training (hard/soft groupDRO and DI).
Key results
- On CIFAR-100, DIM with training-dynamics supervision reaches a cosine similarity of 22.97 vs 3.06 for Jain et al., and detects 57.5% of under-represented subgroups vs 45.0% for Jain et al.
- On Breeds, DIM improves worst-4-subgroup accuracy with DI-based mitigation by about 4.33 points over the next best subgroup discovery baseline (Jain et al. and Domino), with an average gain of 17.3% over ERM.
- On CIFAR-100 data-centric mitigation, DIM improves worst-subgroup accuracy (1st) from 33.7 (Jain et al.) to 35.6, a 3.2% gain over Jain et al.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use training dynamics as supervision for PLS decomposition rather than unsupervised PCA (ablation) | DIM · CIFAR-100 | cosine similarity: 10.76 -> 22.97 | Helped |
| Use training dynamics as supervision for PLS decomposition rather than unsupervised PCA (ablation) | DIM · Breeds | cosine similarity: 7.56 -> 24.15 | Helped |
| Use training dynamics as supervision for subgroup discovery (ablation) | DIM · CIFAR-100 | biased subgroup detection success rate: 52.5 -> 57.5 | Helped |
| Use training dynamics as supervision for subgroup discovery (ablation) | DIM · Breeds | biased subgroup detection success rate: 53.8 -> 61.5 | Helped |
| Filter extra data using DIM-discovered subgroup scores instead of Jain et al. subgroup scores | ResNet-18 · CIFAR-100 test | worst subgroup accuracy (1st): 33.7 -> 35.6 | Helped |
| Use DIM-discovered subgroups for data-centric filtering | ResNet-18 · CIFAR-100 test | overall accuracy: 53.1 -> 54.7 | Helped |
| Use DIM-discovered subgroups with Soft-gDRO | ResNet-18 · CIFAR-100 test | worst subgroup accuracy (2nd): 35.8 -> 38.4 | Helped |
| Use DIM-discovered subgroups for data-centric filtering | ResNet-34 · Breeds test | worst subgroup accuracy (1st): 71.6 -> 72.5 | Helped |
| Use DIM-discovered subgroups with Soft-DI | ResNet-34 · Breeds test | worst subgroup accuracy (1st): 67.3 -> 68.2 | Helped |
| Increase the number of decomposition components from 2 to 9 (ablation) | DIM · CIFAR-100 | cosine similarity: 9.9 -> 28.7 | Helped |
Surprising findings
- Jain et al.'s negative direction contributes only 0.82 of its 3.06 similarity score, so it effectively fails to represent any biased subgroup.
- Without supervision (DIM-PCA), subgroup discovery similarity drops to 10.76, showing the decomposition needs model supervision.
Experimental setup
Datasets: CIFAR-100 Breeds Hard ImageNet LAION-5B
Benchmarks: CIFAR-100 Breeds Hard ImageNet
Metrics: maximum matched cosine similarity biased subgroup detection success rate worst-subgroup accuracy overall accuracy ablation accuracy
Baselines: Jain et al. Domino DIM-PCA JTT SubY LfF EIIL gDRO DI
Builds on: Jain et al. 2023 (failure mode direction discovery) Domino Partial Least Squares groupDRO Domain Independent training
Closest papers by meaning
- Subgroups Matter for Robust Bias Mitigation (ICML 2025 · similarity 0.92)
- Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation (NeurIPS 2026 · similarity 0.86)
- Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness (CVPR 2025 · similarity 0.85)
- Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers (CVPR 2025 · similarity 0.85)
- Improving Subgroup Robustness via Data Selection (NeurIPS 2024 · similarity 0.85)
- Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.