Research map / Label-efficient and robust learning / Knowledge distillation
Self-Training With Noisy Student Improves ImageNet Classification
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Noisy Student Training with unlabeled images | EfficientNet-L2 · ImageNet 2012 validation | top-1 accuracy: 85.5% -> 88.4% | Helped |
| Apply Noisy Student Training | EfficientNet-L2 · ImageNet-A | top-1 accuracy: 49.6% -> 83.7% | Helped |
| Apply Noisy Student Training | EfficientNet-L2 · ImageNet-C | mCE: 45.7 to 28.3 | Helped |
| Apply Noisy Student Training | EfficientNet-L2 · ImageNet-P | mFR: 27.8 to 12.2 | Helped |
| Add augmentation noise to unlabeled student training (ablation) | EfficientNet-B5 Noisy Student Training · ImageNet 2012 validation, 130M unlabeled images | top-1 accuracy: 84.6% -> 85.1% | Helped |
| Add augmentation, stochastic depth, and dropout noise to unlabeled student training (ablation) | EfficientNet-B5 Noisy Student Training · ImageNet 2012 validation, 130M unlabeled images | top-1 accuracy: 84.3% -> 85.1% | Helped |
| Add noise to the teacher when generating pseudo labels (ablation) | EfficientNet-B5 Noisy Student Training · ImageNet 2012 validation, 130M unlabeled images | top-1 accuracy: 85.1% -> 84.4% | Hurt |
| Iterate teacher-student training (ablation) | EfficientNet-L2 · ImageNet 2012 validation | top-1 accuracy: 87.6% -> 88.1% -> 88.4% | Helped |
| Use an EfficientNet-L2 teacher instead of a same-size teacher (ablation) | Noisy Student Training with EfficientNet-B0 student · ImageNet 2012 validation | top-1 accuracy: 78.1% -> 78.8% | Helped |
| Increase student size from EfficientNet-B4 to EfficientNet-B5 (ablation) | Noisy Student Training with EfficientNet-B4 teacher · ImageNet 2012 validation | top-1 accuracy: 84.0% -> 84.7% | Helped |
| Use more unlabeled data, from 1/128 to the full set (ablation) | Noisy Student Training with EfficientNet-B4 · ImageNet 2012 validation | top-1 accuracy: 83.4% -> 84.0% | Mixed |
| Use soft instead of hard pseudo labels for low-confidence, out-of-domain images (ablation) | Noisy Student Training · ImageNet 2012 validation | top-1 accuracy | Helped |
Closest papers by meaning
- SELFIE: Refurbishing Unclean Samples for Robust Deep Learning (ICML 2019 · similarity 0.88)
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.87)
- Meta Pseudo Labels (CVPR 2021 · similarity 0.87)
- Doubly-Robust Self-Training (NeurIPS 2023 · similarity 0.87)
- Big Self-Supervised Models are Strong Semi-Supervised Learners (NeurIPS 2020 · similarity 0.86)
- Toward Improving Robustness and Accuracy in Unsupervised Domain Adaptation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.