Research map / Label-efficient and robust learning / Recognition and long-tail
Classes Are Not Equal: An Empirical Study on Image Recognition Fairness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Empirical study showing extreme per-class accuracy disparity on balanced datasets (e.g., ImageNet, CIFAR-100, WIT-400M), attributes it to problematic representation and model prediction bias, and tests augmentation, representation learning and re-weighting as fairness remedies.
Problem
Image classifiers show extreme per-class accuracy disparity even on class-balanced datasets such as ImageNet, and the causes of this unfairness are not well understood beyond long-tailed class frequency effects.
Approach
Distinguishes image recognition unfairness from long-tailed recognition: shows the disparity stems from problematic representation rather than classifier bias, and introduces Model Prediction Bias (false-positive frequency) with a gradient-convergence lemma to explain why hard classes get low accuracy on balanced data.
Key results
- ResNet-50 on ImageNet reaches 100% top-1 on the best class (ostrich) but only 16% on the worst class (screen), with Easy/Medium/Hard averages of 93.1/81.1/59.4.
- On ImageNet, ViT-B with all augmentations (AutoAug, Mixup, CutMix) raises Hard-subset accuracy from 64.2 to 68.9 (+4.7) versus +1.4 on Easy, and overall accuracy from 80.8 to 83.6.
- Re-weighting improves Hard-class accuracy (+2.0 on ImageNet) but lowers Easy-class accuracy (-1.4) and overall accuracy (77.8 to 77.6).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AutoAug, Mixup, and CutMix | ViT-B · ImageNet | top-1 accuracy (Hard): 64.2 -> 68.9 (+4.7) | Helped |
| add CutMix | WideResNet-34-10 · CIFAR-100 | top-1 accuracy (Hard): 70.1 -> 73.9 (+3.8) | Helped |
| add AutoAug | WideResNet-34-10 · CIFAR-100 | top-1 accuracy (Easy): 92.2 -> 91.7 (-0.5) | Hurt |
| add Mixup | WideResNet-34-10 · CIFAR-100 | top-1 accuracy (Easy): 92.2 -> 91.3 (-0.9) | Hurt |
| add contrastive pre-training | ResNet-50 · ImageNet | top-1 accuracy (Hard): 62.0 -> 62.2 (+0.2) | Helped |
| use GPaCo | ResNet-50 · ImageNet | top-1 accuracy (Hard): 62.0 -> 62.6 (+0.6) | Helped |
| use masked-modeling pre-training | ViT-B · ImageNet | top-1 accuracy (Hard): 64.8 -> 68.9 (+4.1) | Helped |
| combine GPaCo and masked-modeling pre-training | ViT-B · ImageNet | top-1 accuracy (Hard): 64.8 -> 69.1 (+4.3) | Helped |
| apply re-weighting [31] | ResNet-50 · ImageNet | top-1 accuracy (Hard): 59.4 -> 61.4 (+2.0) | Helped |
| apply re-weighting [31] | ResNet-50 · ImageNet | top-1 accuracy (All): 77.8 -> 77.6 | Hurt |
| apply LfF | WideResNet-34 · CIFAR-100 | top-1 accuracy (Hard): 70.1 -> 67.7 (-2.4) | Hurt |
| evaluate with ImageNet-ReaL multi-label annotations | ResNet-50 · ImageNet-ReaL | worst-class top-1 accuracy: 16% -> 45% | Helped |
Surprising findings
- Classes with lower accuracy show larger class-mean separation angles, contrary to the common belief that larger separation correlates with higher accuracy.
- Classifier weight norms are already balanced on ImageNet models despite accuracy ranging from 100% to 16%, so classifier norm is not the core cause of unfairness.
Limitations
- Analysis focuses on empirical correlations and a single-label setup; multi-label co-occurrence effects are only partly explored via ImageNet-ReaL.
- No dedicated fairness-promoting method is proposed; improvements come from existing techniques.
- Hard/Medium/Easy partitions depend on a reference model's per-class performance.
Experimental setup
Datasets: ImageNet CIFAR-100 ImageNet-LT WIT-400M ImageNet-ReaL
Benchmarks: ImageNet CIFAR-100 ImageNet-LT WIT-400M ImageNet-ReaL
Metrics: top-1 accuracy per-class accuracy disparity (Easy/Medium/Hard subsets) best and worst class accuracy prediction bias
Baselines: Mixup CutMix AutoAugment RandAugment MoCo GPaCo MAE re-weighting LfF JTT
Builds on: Mixup CutMix AutoAugment MAE GPaCo decoupled classifier learning (cRT)
Closest papers by meaning
- Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers (CVPR 2022 · similarity 0.89)
- FairDD: Fair Dataset Distillation (NeurIPS 2025 · similarity 0.87)
- FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA (NeurIPS 2025 · similarity 0.87)
- Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance (ICML 2024 · similarity 0.87)
- Leveraging Diffusion Perturbations for Measuring Fairness in Computer Vision (AAAI 2024 · similarity 0.87)
- Kernel-based Equalized Odds: A Quantification of Accuracy-Fairness Trade-off in Fair Representation Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.