Research map / Label-efficient and robust learning / Recognition and long-tail
From ImageNet to Image Classification: Contextualizing Progress on Benchmarks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate predictions against any human-identified valid object label instead of only the single ImageNet label | ImageNet-trained models · ImageNet multi-object images | accuracy: more than 10% | Helped |
| Ask annotators to choose among multiple candidate labels simultaneously rather than validate each label in isolation | ImageNet human annotation · ImageNet validation images | number of labels selected | Helped |
| Use ImageNet top-5 accuracy instead of top-1 accuracy to account for multi-object images | ImageNet-trained models · ImageNet multi-object images | accuracy validity: about 20% | Hurt |
| Increase ImageNet model accuracy | ImageNet-trained models · ImageNet | human agreement with predictions | Helped |
| Subsample the annotator pool to 5 rather than 9 annotators (ablation) | ImageNet selection-frequency estimates · ImageNet validation images | conclusions about label-selection frequency: 5 instead of 9 annotators | No clear effect |
Closest papers by meaning
- Does progress on ImageNet transfer to real-world datasets? (NeurIPS 2023 · similarity 0.89)
- SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification (NeurIPS 2024 · similarity 0.89)
- When does dough become a bagel? Analyzing the remaining mistakes on ImageNet (NeurIPS 2022 · similarity 0.89)
- Does Progress On Object Recognition Benchmarks Improve Generalization on Crowdsourced, Global Data? (ICLR 2024 · similarity 0.88)
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.87)
- A Decade's Battle on Dataset Bias: Are We There Yet? (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.