AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

Do ImageNet Classifiers Generalize to ImageNet?

Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, Vaishaal Shankar

ICML 2019 · Label-efficient and robust learning › Recognition and long-tail · approach: Loss design and classifier theory · 541 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Evaluate existing classifiers on newly collected CIFAR-10 imagesCIFAR-10 classifiers · CIFAR-10.1accuracy: 98.4 -> 95.5Hurt
Evaluate existing classifiers on newly collected CIFAR-10 imagesCIFAR-10 classifiers · CIFAR-10.1accuracy: 97.1 -> 93.0Hurt
Evaluate existing classifiers on newly collected ImageNet imagesImageNet classifiers · ImageNetV2 MatchedFrequencytop-1 accuracy: 82.9 -> 72.2Hurt
Evaluate existing classifiers on newly collected ImageNet imagesImageNet classifiers · ImageNetV2 MatchedFrequencytop-1 accuracy: 82.5 -> 72.2Hurt
Increase the average MTurk selection frequency of sampled images using TopImages instead of MatchedFrequencyImageNet classifiers · ImageNetV2average top-1 accuracy change: -11.8% -> +2.1%Improved
Use Threshold0.7 sampling rather than MatchedFrequencyImageNet classifiers · ImageNetV2average top-1 accuracy change: -11.8% -> -3.2%Improved
Increase original-test accuracy across modelsCIFAR-10 classifiers · CIFAR-10.1accuracy: slope 1.69Improved
Increase original-test accuracy across modelsImageNet classifiers · ImageNetV2 MatchedFrequencytop-1 accuracy: slope 1.11Improved
Retune learning rate, dropout, and weight decay for VGGvgg16_keras · CIFAR-10.1accuracy: 85.3% -> 85.8%No clear effect
Add about 1,000 CIFAR-10.1 images to the training setvgg16_keras · Held-out CIFAR-10.1accuracy: 84.9% -> 85.1% and 85.4%No clear effect
Evaluate human accuracy on the hardest images from the new rather than original CIFAR-10 test setHuman participants · Hardest CIFAR-10 imagesaverage accuracy difference: -0.8%No clear effect
Train an ℓ∞-robust CIFAR-10 model and evaluate it on the new test setℓ∞-robust CIFAR-10 model · CIFAR-10.1accuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy