Research map / Label-efficient and robust learning / Knowledge distillation
When does label smoothing help?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with label smoothing alpha=0.1 | ResNet-56 · CIFAR-100 | top-1 accuracy: 72.1 -> 72.7 | Helped |
| train with label smoothing alpha=0.1 | AlexNet · CIFAR-10 | top-1 accuracy: 86.8 -> 86.7 | No clear effect |
| train with label smoothing alpha=0.1 | Inception-v4 · ImageNet | top-1 accuracy: 80.9 -> 80.9 | No clear effect |
| train with label smoothing | ResNet-56 · CIFAR-100 | ECE: 0.150 -> 0.024 | Helped |
| train with label smoothing | Inception-v4 · ImageNet | ECE: 0.071 -> 0.035 | Helped |
| train with label smoothing | Transformer · WMT EN-DE | ECE: 0.056 -> 0.019 | Helped |
| train with label smoothing | Transformer · WMT EN-DE | NLL | Hurt |
| apply temperature scaling to hard-target model instead of label smoothing | Transformer · WMT EN-DE | BLEU | Hurt |
| train teacher with label smoothing instead of dropout | knowledge distillation · MNIST | student test error: 0.74 -> 0.91 | Hurt |
| train teacher with label smoothing | ResNet-56 -> AlexNet distillation · CIFAR-10 | student accuracy | Hurt |
| train with label smoothing | AlexNet student (no distillation) · CIFAR-10 | accuracy | Hurt |
Closest papers by meaning
- MaxSup: Overcoming Representation Collapse in Label Smoothing (NeurIPS 2025 · similarity 0.88)
- Does Label Smoothing Help Deep Partial Label Learning? (ICML 2024 · similarity 0.87)
- Training Deep Neural Networks with Virtual Smoothing Classes (AAAI 2025 · similarity 0.87)
- Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It (ICLR 2025 · similarity 0.87)
- Calibrated Language Models and How to Find Them with Label Smoothing (ICML 2025 · similarity 0.87)
- A Label is Worth A Thousand Images in Dataset Distillation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.