Research map / Label-efficient and robust learning / Recognition and long-tail
Attention IoU: Examining Biases in CelebA using Attention Maps
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Attention-IoU, a size- and scale-invariant IoU over GradCAM attention maps, to measure model bias via mask and heatmap scores, and applies it to Waterbirds and CelebA.
Problem
Existing bias metrics for image classifiers rely on dataset label distributions and subgroup accuracy, and cannot reveal which image regions drive spurious correlations inside a model.
Approach
A generalized IoU (Attention-IoU) computed on L1-normalized dense maps lets attention maps be compared with ground-truth feature masks (mask score) or with other attributes' attention maps (heatmap score), using GradCAM on the absolute class output for binary classifiers.
Key results
- On Waterbirds, mask score falls from 0.72 ± 0.02 to 0.42 ± 0.03 as training bias rises from 50% to 100%, mirroring the drop in worst group accuracy (0.81 to 0.21).
- On CelebA, the heatmap score for Male vs. attributes ranges from 0.63 ± 0.02 (Black Hair) to 0.94 ± 0.01 (Wearing Lipstick), with a positive trend against MCC.
- Subsampling experiments show Blond Hair heatmap score is unchanged when its training correlation with Male is varied (Kendall ω 0.007), while Wavy Hair's changes strongly (ω 0.785), suggesting a hidden confounder for Blond Hair.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase training-set bird-background bias from 50% to 100% | ResNet-18 trained on Waterbirds · Waterbirds test set | worst-group accuracy: 0.81±0.02 to 0.21±0.10 | Hurt |
| Increase training-set bird-background bias from 50% to 100% | Attention-IoU mask score with ResNet-18 · Waterbirds test set, bird mask | Attention-IoU mask score: 0.72 ± 0.02 to 0.42 ± 0.03 | Hurt |
| Vary the training-set correlation between Blond Hair and Male by subsampling | Attention-IoU (GradCAM), Blond Hair · CelebA subsampled training set | heatmap score with Male: Kendall ω=0.007 | No clear effect |
| Vary the training-set correlation between Wavy Hair and Male by subsampling | Attention-IoU (GradCAM), Wavy Hair · CelebA subsampled training set | heatmap score with Male: ω=0.785 | Mixed |
Surprising findings
- Blond Hair's attention overlap with Male does not change when the dataset correlation with Male is reduced, suggesting an unlabeled confounder rather than label correlation.
- Eyeglasses has a high heatmap score (0.86) despite a low MCC with Male (0.26), attributed to co-localized features rather than a true bias.
Limitations
- Attention maps convey only spatial information; co-located attributes with different features cannot be distinguished, causing false high scores (e.g., Eyeglasses).
- Mask score analysis is limited to 17 CelebA attributes with corresponding feature masks in CelebAMask-HQ.
- Uses binary labels for CelebA's Male attribute, which the authors acknowledge forces gender presentation into binaries.
Experimental setup
Datasets: Waterbirds CUB Places CelebA CelebAMask-HQ ImageNet
Benchmarks: Waterbirds CelebA CelebAMask-HQ
Metrics: Attention-IoU mask score Attention-IoU heatmap score worst group accuracy Matthews correlation coefficient Kendall omega
Baselines: worst group accuracy pointing game intersection-over-union
Builds on: GradCAM Group DRO
Closest papers by meaning
- Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers (CVPR 2025 · similarity 0.87)
- Common Sense Bias Modeling for Classification Tasks (AAAI 2025 · similarity 0.85)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.85)
- Discovering and Mitigating Visual Biases through Keyword Explanation (CVPR 2024 · similarity 0.84)
- Understanding Bias in Large-Scale Visual Datasets (NeurIPS 2024 · similarity 0.84)
- ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.