Research map / Label-efficient and robust learning / Noisy labels
Automated Classification of Model Errors on ImageNet
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-label accuracy rather than top-1 accuracy as the reference for model comparison | ImageNet model error analysis · ImageNet validation | portion of severe model failures among errors | Helped |
| Increase pre-training dataset size | ImageNet classifiers at comparable MLA · ImageNet validation, organisms and artifacts | portion of model failures | Helped |
| Increase model size while conditioning on MLA | CNN and transformer classifiers · ImageNet validation | portion of model failures | No clear effect |
| Use transformer-based models rather than ConvNeXts | ImageNet classifiers · ImageNet validation, organisms | rate of model-failure reduction with MLA | Mixed |
| Apply automated categorization to ImageNet-A | ImageNet error-classification pipeline · ImageNet-A | portion of errors explained for best-performing models: up to half | Hurt |
| Use ImageNet MLA to track progress on ImageNet-A | ImageNet-A model evaluation · ImageNet-A | portion of model failures | Helped |
| Use a pipeline that selects the least severe explanation when multiple apply | Automated error classification · ImageNet validation, ViT-3B errors | agreement with human expert annotations: 73.1% | Helped |
Closest papers by meaning
- When does dough become a bagel? Analyzing the remaining mistakes on ImageNet (NeurIPS 2022 · similarity 0.93)
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.91)
- Evaluating Machine Accuracy on ImageNet (ICML 2020 · similarity 0.89)
- Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks (NeurIPS 2021 · similarity 0.88)
- Making Better Mistakes: Leveraging Class Hierarchies With Deep Networks (CVPR 2020 · similarity 0.88)
- ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations (ICLR 2023 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.