Research map / Label-efficient and robust learning / Recognition and long-tail
Common Sense Bias Modeling for Classification Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Common Sense Bias Discovery (CSBD), which clusters noun-chunk embeddings from image descriptions into features, measures pairwise phi correlations to find spurious correlations, and mitigates them by re-sampling weights, outperforming unsupervised debiasing baselines on CelebA and MS-COCO.
Problem
Dataset composition creates spurious feature correlations that bias downstream image classifiers, and existing unsupervised de-biasing methods only address prominent bias features such as digit colour or background, missing broader common-sense correlations.
Approach
Discover bias from text descriptions rather than model performance: noun chunks are clustered by sentence-embedding similarity into features, feature co-occurrence is measured with phi coefficients, a human can optionally review pairs, and a re-sampling scheme decorrelates the selected features from the target without sensitive labels.
Key results
- On CelebA, CSBD raises worst-group accuracy for Young/Eyeglasses from 0.652 (no mitigation) to 0.655, and for Receding hairline/Grey hair from 0.186 to 0.337 (BPA 0.351 is best there).
- On MS-COCO, CSBD achieves the best worst-group accuracy on most target-spurious pairs, e.g. TV/Chair 0.726 to 0.842 and Umbrella/Person 0.761 to 0.814.
- Correlation between description-derived bias and label-derived bias is 0.781 (95% CI 0.417 to 1.0), while feature-presence vs label correlation is only 0.545.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CSBD-derived feature correlations to reweight training data | ResNet50 classification · MS-COCO object classification tasks | worst-group accuracy | Helped |
| Use CSBD-derived feature correlations to reweight training data | ResNet50 classification · CelebA attribute classification tasks | average group accuracy | Mixed |
| Vary the clustering distance threshold z from 0.5 to 1.5 (ablation) | CSBD · CelebA: Eyeglasses / Man | worst-group accuracy: 0.971 to 0.973 | No clear effect |
| Double the increased resampling weight (ablation) | CSBD · MS-COCO: Cat / Couch | worst-group accuracy: Wst.: 0.904 | No clear effect |
| Halve the increased resampling weight (ablation) | CSBD · MS-COCO: Cat / Couch | worst-group accuracy: Wst.: 0.881 | Hurt |
Surprising findings
- DebiAN, PGD and BPA mix samples within a target class without fixing the decision boundary for cat classification on MS-COCO.
- LVLM-generated (LLaVA-Next) captions yield bias discovery and mitigation results broadly consistent with human-written captions.
Limitations
- Discovered biases are limited to features that appear in textual descriptions.
- Only pairwise feature correlations are studied; higher-order correlations are not explored.
- Method is not tested on detection or segmentation tasks.
Experimental setup
Datasets: CelebA-Dialog CelebA MS-COCO 2014
Benchmarks: CelebA MS-COCO 2014
Metrics: worst-group accuracy average accuracy phi correlation coefficient SPICE
Baselines: LfF DebiAN PGD BPA
Builds on: Universal Sentence Encoder LfF DebiAN PGD BPA
Closest papers by meaning
- Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers (CVPR 2025 · similarity 0.89)
- Understanding Bias in Large-Scale Visual Datasets (NeurIPS 2024 · similarity 0.88)
- Discovering and Mitigating Visual Biases through Keyword Explanation (CVPR 2024 · similarity 0.88)
- Unbiasing through Textual Descriptions: Mitigating Representation Bias in Video Benchmarks (CVPR 2025 · similarity 0.86)
- Attention IoU: Examining Biases in CelebA using Attention Maps (CVPR 2025 · similarity 0.85)
- TRoVe: Discovering Error-Inducing Static Feature Biases in Temporal Vision-Language Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.