Research map / Adversarial robustness and security / Certified and empirical robustness
Measuring Robustness to Natural Distribution Shifts in Image Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on substantially more and more diverse data | ImageNet classifiers · Natural dataset shifts | effective robustness: ρ = 2.1\%; ρ = 1.5\%; ρ = 1.1\% | Helped |
| Train on substantially more data | EfficientNet-L2 (NoisyStudent) · ImageNetV2 | accuracy drop: 8.6\% to 7.5\% | Helped |
| Apply ℓp adversarial training | ImageNet classifiers · ImageNet-Vid-Robust and YTBB-Robust consistency shifts | effective robustness: average ρ = 6.7\%; average ρ = 4.9\% | Helped |
| Apply ℓp adversarial training | ResNet50 · ImageNet-Vid-Robust | relative robustness: average τ = -8.5\% | Hurt |
| Use corruption accuracy as a proxy for natural robustness | ImageNet models · ImageNetV2 | correlation with effective robustness: r = 0.24 | No clear effect |
| Use ℓp attack robustness as a proxy for natural robustness | ImageNet models · ImageNetV2 | correlation with effective robustness: r = -0.05 | No clear effect |
| Vary the number of images per class through i.i.d. subsampling (ablation) | ILSVRC-2012-trained models · ImageNetV2 | effective robustness | No clear effect |
| Vary the number of training classes (ablation) | ILSVRC-trained models · 125-class ImageNet evaluation | effective robustness | No clear effect |
| Increase training data using JFT-300M | ResNet101 · ImageNetV2 | effective robustness: ρ = -0.23\% | No clear effect |
Closest papers by meaning
- On Robustness and Transferability of Convolutional Neural Networks (CVPR 2021 · similarity 0.90)
- A Flat Minima Perspective on Understanding Augmentations and Model Robustness (AAAI 2026 · similarity 0.89)
- Towards Robustness to Natural Variations and Distribution Shift (Student Abstract) (AAAI 2024 · similarity 0.89)
- Unexplored Faces of Robustness and Out-of-Distribution: Covariate Shifts in Environment and Sensor Domains (CVPR 2024 · similarity 0.88)
- ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object (CVPR 2024 · similarity 0.86)
- OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shift (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.