Research map / Label-efficient and robust learning / OOD and anomaly detection
Can Biases in ImageNet Models Explain Generalization?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Large-scale study of 48 ResNet-50 ImageNet models with different training methods, measuring shape bias, spectral (low/high-frequency) biases, and critical band against four generalization categories, finding no single bias explains generalization holistically.
Problem
Whether known model-vs-human biases (texture/shape bias, spectral bias, critical band) actually explain and predict the generalization of ImageNet classifiers across ID, corruption, conceptual, and adversarial benchmarks.
Approach
Fix the architecture to ResNet-50, vary only the training recipe across 48 models, and re-measure biases and generalization under a consistent, system-controlled evaluation pipeline to test whether previously reported bias-generalization correlations hold holistically.
Key results
- Shape bias shows a strong monotonic negative correlation with ID accuracy (ImageNet) without significant outliers, while adversarially-trained models show a curvilinear relation to other generalization aspects.
- Center frequency of the critical band shows no correlation with any benchmark, contrary to the original study.
- ImageNet-bar-C shows the strongest average correlation (r = 0.83) with the remaining benchmarks, versus r = 0.79 for ImageNet accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase shape bias | ImageNet ResNet-50 models · ImageNet ID tests | accuracy | Hurt |
| Use a stronger texture bias | ImageNet ResNet-50 models · ImageNet-A | accuracy | Helped |
| Balance shape and texture bias | Adversarially trained ResNet-50 models · Concepts and adversarial robustness | accuracy | Helped |
| Increase low-frequency bias | ImageNet ResNet-50 models · Adversarial robustness (PGD) | accuracy | Helped |
| Increase low-frequency bias | ImageNet ResNet-50 models · Generalization beyond adversarial robustness | accuracy | No clear effect |
| Increase low-frequency bias | Adversarially trained ResNet-50 models · Concepts and adversarial robustness | accuracy | Mixed |
| Increase high-frequency bias | ImageNet ResNet-50 models · Generalization except adversarial robustness | accuracy | Mixed |
| Increase high-frequency bias | ImageNet ResNet-50 models · Adversarial robustness | accuracy | Hurt |
| Increase critical-band bandwidth | ImageNet ResNet-50 models · ImageNet ID and robustness benchmarks | accuracy | Mixed |
| Increase critical-band bandwidth | ImageNet ResNet-50 models · Adversarial robustness | accuracy | Mixed |
| Change critical-band center frequency | ImageNet ResNet-50 models · Generalization benchmarks | accuracy | No clear effect |
| Use peak-noise sensitivity as a generalization indicator | ImageNet ResNet-50 models · Generalization benchmarks | accuracy | No clear effect |
Surprising findings
- High-frequency bias correlates positively with most generalization aspects despite high-pass filtered images carrying little discriminative content; the strongest high-frequency-biased model detects only about 9% of samples from that band alone.
- Shape bias is not determined by dataset scale: a self-supervised ResNet-50 trained on 100M extra YFCC images has the same 0.21 shape bias as the baseline.
Limitations
- Limited to ResNet-50 for main analysis; adversarially-trained models are homogeneous (same hyperparameters except adversary parameters), so AT-driven correlations may be spurious.
- Correlation does not imply causation; observed correlations cannot be causally confirmed with this model zoo.
- Peak-noise sensitivity measurements concentrate around 1, suggesting the test may not capture trends well at scale.
Experimental setup
Datasets: ImageNet ImageNet-v2 ImageNet-ReaL ImageNet-C ImageNet-bar-C ImageNet-A ImageNet-R ImageNet-Sketch Stylized ImageNet cue-conflict dataset
Benchmarks: ImageNet ImageNet-v2 ImageNet-ReaL ImageNet-C ImageNet-bar-C ImageNet-A ImageNet-R ImageNet-Sketch Stylized ImageNet PGD adversarial attack cue-conflict
Metrics: top-1 accuracy Spearman rank correlation shape bias low/high-frequency bias ratio critical band bandwidth, center frequency, peak-noise sensit
Baselines: ResNet-50 baseline (He et al.) ShapeNet / SIN-trained model DeepAugment+AugMix TIMM A3 recipe adversarially trained ResNet-50 models
Builds on: ImageNet-trained shape bias study (Geirhos et al.) critical band study (Subramanian et al.) high-frequency bias study (Wang et al.) PGD adversarial training
Closest papers by meaning
- Biased Generalization in Diffusion Models (ICML 2026 · similarity 0.91)
- Simplicity Bias in Overparameterized Machine Learning (AAAI 2024 · similarity 0.87)
- Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks (ICML 2024 · similarity 0.87)
- Position: Deep Learning is Not So Mysterious or Different (ICML 2025 · similarity 0.86)
- What Variables Affect Out-of-Distribution Generalization in Pretrained Models? (NeurIPS 2024 · similarity 0.86)
- The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.