Research map / Label-efficient and robust learning / Noisy labels
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate ResNet-18 on corrected rather than original labels | ResNet-18 · ImageNet correctable set | top-1 accuracy: 6.51 -> 82.42 | Helped |
| Evaluate ResNet-50 on corrected rather than original labels | ResNet-50 · ImageNet correctable set | top-1 accuracy: 13.52 -> 73.74 | Helped |
| Evaluate VGG-11-BN on corrected rather than original labels | VGG-11-BN · CIFAR-10 correctable set | top-1 accuracy: 27.78 -> 66.67 | Helped |
| Evaluate VGG-19-BN on corrected rather than original labels | VGG-19-BN · CIFAR-10 correctable set | top-1 accuracy: 50.00 -> 38.89 | Hurt |
| Prefer lower-capacity ResNet-18 over ResNet-50 when evaluating corrected labels | ImageNet models · ImageNet correctable set | corrected top-1 accuracy: 82.42 vs 73.74 | Helped |
| Prefer lower-capacity VGG-11-BN over VGG-19-BN when evaluating corrected labels | CIFAR-10 models · CIFAR-10 correctable set | corrected top-1 accuracy: 66.67 vs 38.89 | Helped |
| Increase originally mislabeled test-example prevalence | ImageNet model selection · ImageNet | corrected top-1 accuracy ranking: N = 9%; increases by around 6% relative to N = 2.9% | Mixed |
| Use top-5 rather than top-1 accuracy for the correctable subset | ImageNet models · ImageNet correctable set | correlation between original-label and corrected-label accuracy | Mixed |
Closest papers by meaning
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.85)
- Object Detection Benchmarks are Incomplete: The Role of Label Errors and Annotation Uncertainty (NeurIPS 2026 · similarity 0.85)
- LEMoN: Label Error Detection using Multimodal Neighbors (ICML 2025 · similarity 0.85)
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.85)
- Revisiting Interpolation for Noisy Label Correction (AAAI 2025 · similarity 0.84)
- Regretful Decisions under Label Noise (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.