Research map / Label-efficient and robust learning / Recognition and long-tail
Does progress on ImageNet transfer to real-world datasets?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use higher ImageNet-accuracy architectures after VGG | ImageNet-pretrained models fine-tuned on four real-world datasets · APTOS, HPA, Melanoma, and Cassava | normalized slope of downstream performance versus ImageNet accuracy: less than 0.05 | No clear effect |
| Use higher ImageNet-accuracy architectures after VGG | ImageNet-pretrained models fine-tuned on CCT-20 · CCT-20 | normalized slope: 0.11 | Helped |
| Use higher ImageNet-accuracy architectures after VGG | ImageNet-pretrained models fine-tuned on HPA · Human Protein Atlas | normalized slope: 0.29 | Helped |
| Increase color and affine augmentation (ablation) | ResNet-50 on APTOS · APTOS 2019 Blindness | quadratic weighted kappa: 0.03 | Helped |
| Use more augmentation (ablation) | ResNet-50 on APTOS · APTOS 2019 Blindness | quadratic weighted kappa: 0.8669 -> 0.8999 | Helped |
| Fine-tune with RandAugment rather than base augmentation (ablation) | ResNet-50 pretrained with base augmentation · CCT-20 | accuracy: 72.02 -> 73.57 | Helped |
| Fine-tune with RandAugment rather than base augmentation (ablation) | ResNet-50 pretrained with base augmentation · APTOS | quadratic weighted kappa: 0.9210 -> 0.9250 | Helped |
| Pre-train with AugMix rather than base augmentation (ablation) | ResNet-50 on APTOS · APTOS | quadratic weighted kappa: 0.9210 -> 0.9239 | No clear effect |
| Pre-train with RandAugment rather than base augmentation (ablation) | ResNet-50 on CCT-20 and APTOS · CCT-20 and APTOS | accuracy / quadratic weighted kappa: 72.02 -> 72.94; 0.9210 -> 0.9190 | Mixed |
| Fine-tune DeiT models with additional augmentation (ablation) | DeiT-tiny and DeiT-small · CCT-20 and APTOS | accuracy / quadratic weighted kappa | No clear effect |
| Use CLIP pre-training instead of ImageNet pre-training with matched architecture | ResNet-50 and ViT-B/16 · CCT-20, APTOS, HPA, Melanoma, Cassava, and EuroSAT | dataset-specific task metric | Mixed |
| Use CLIP-L14@336 rather than the best ImageNet-pretrained model | Fine-tuning on downstream datasets · CCT-20, APTOS, HPA, Melanoma, Cassava, and EuroSAT | dataset-specific task metric | Mixed |
Closest papers by meaning
- Do Better ImageNet Models Transfer Better? (CVPR 2019 · similarity 0.92)
- From ImageNet to Image Classification: Contextualizing Progress on Benchmarks (ICML 2020 · similarity 0.89)
- SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification (NeurIPS 2024 · similarity 0.86)
- Effective pruning of web-scale datasets based on complexity of concept clusters (ICLR 2024 · similarity 0.85)
- Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models (ICML 2024 · similarity 0.85)
- ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracy (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.