Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows empirically that label smoothing consistently degrades selective classification across ImageNet, Cityscapes and small-scale tasks, explains this via logit-level gradient analysis (imbalanced max-logit suppression), and shows post-hoc logit normalisation recovers the lost SC performance.
Problem
Label smoothing improves accuracy but has been observed to degrade selective classification (rejecting misclassifications using model uncertainty), with no clear explanation of why, and why logit normalisation helps is unclear.
Approach
Analyses logit-level gradients of the LS loss to show LS suppresses the max logit more when a prediction is likely correct and less when likely wrong, degrading the uncertainty ranking; post-hoc p-norm logit normalisation reverses this by increasing uncertainty for high max-logit samples.
Key results
- Label smoothing consistently degrades selective classification on ImageNet (ResNet-50, ViT-S-16) and Cityscapes, with stronger alpha giving greater degradation; at 1% risk target, all LS models have close to zero coverage.
- At 10% coverage on ImageNet+ImageNet-Sketch, ResNet-50 error rate on sketch samples rises from 6.4% (CE) to 27.4% (LS alpha=0.3).
- Post-hoc logit normalisation greatly improves SC of LS-trained models while having little effect on CE-trained models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with label smoothing α=0.1 instead of CE | ResNet-50 · ImageNet + ImageNet-Sketch, ImageNet subset at 10% coverage | error rate: 1.2 -> 2.3 | Hurt |
| increase label smoothing from α=0.1 to α=0.2 | ResNet-50 · ImageNet + ImageNet-Sketch, ImageNet subset at 10% coverage | error rate: 2.3 -> 3.3 | Hurt |
| increase label smoothing from α=0.2 to α=0.3 | ResNet-50 · ImageNet + ImageNet-Sketch, ImageNet subset at 10% coverage | error rate: 3.3 -> 3.8 | Hurt |
| train with label smoothing α=0.1 instead of CE | ResNet-50 · ImageNet + ImageNet-Sketch, Sketch subset at 10% coverage | error rate: 6.4 -> 17.8 | Hurt |
| increase label smoothing from α=0.1 to α=0.2 | ResNet-50 · ImageNet + ImageNet-Sketch, Sketch subset at 10% coverage | error rate: 17.8 -> 24.1 | Hurt |
| increase label smoothing from α=0.2 to α=0.3 | ResNet-50 · ImageNet + ImageNet-Sketch, Sketch subset at 10% coverage | error rate: 24.1 -> 27.4 | Hurt |
| train with label smoothing α=0.1 instead of CE | ViT-S-16 · ImageNet + ImageNet-Sketch, ImageNet subset at 10% coverage | error rate: 1.1 -> 1.9 | Hurt |
| increase label smoothing from α=0.1 to α=0.2 | ViT-S-16 · ImageNet + ImageNet-Sketch, ImageNet subset at 10% coverage | error rate: 1.9 -> 2.5 | Hurt |
| increase label smoothing from α=0.2 to α=0.3 | ViT-S-16 · ImageNet + ImageNet-Sketch, ImageNet subset at 10% coverage | error rate: 2.5 -> 3.3 | Hurt |
| train with label smoothing α=0.1 instead of CE | ViT-S-16 · ImageNet + ImageNet-Sketch, Sketch subset at 10% coverage | error rate: 6.3 -> 11.9 | Hurt |
| increase label smoothing from α=0.1 to α=0.2 | ViT-S-16 · ImageNet + ImageNet-Sketch, Sketch subset at 10% coverage | error rate: 11.9 -> 13.9 | Hurt |
| increase label smoothing from α=0.2 to α=0.3 | ViT-S-16 · ImageNet + ImageNet-Sketch, Sketch subset at 10% coverage | error rate: 13.9 -> 20.0 | Hurt |
Surprising findings
- Label smoothing degrades selective classification even though it improves top-1 accuracy at full coverage.
- Logit normalisation has little effect on cross-entropy-trained models, whose max logit carries little information about error probability given MSP.
Limitations
- Explanation is based on training-loss gradients under an idealised analysis that assumes the empirical loss approximates the true loss.
- Small-scale tabular and text experiments produce noisy risk-coverage estimates and show a less pronounced effect; CIFAR-100 LS did not improve top-1 error.
Experimental setup
Datasets: ImageNet-1k ImageNet-Sketch Cityscapes CIFAR-100 CIFAR-10 Bank Marketing Online Shoppers Intentions IMDB
Benchmarks: ImageNet-1k ImageNet-Sketch Cityscapes CIFAR-100 Bank Marketing Online Shoppers IMDB
Metrics: AURC Cov@5% risk coverage at 1% risk risk-coverage curve top-1 error
Baselines: cross entropy training maximum softmax probability DOCTOR Energy score
Builds on: label smoothing logit normalisation sharpness-aware minimisation TorchUncertainty
Closest papers by meaning
- MaxSup: Overcoming Representation Collapse in Label Smoothing (NeurIPS 2025 · similarity 0.90)
- Does Label Smoothing Help Deep Partial Label Learning? (ICML 2024 · similarity 0.86)
- Robust Logit Adjustment for Learning with Long-Tailed Noisy Data (AAAI 2025 · similarity 0.85)
- Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks (ICLR 2024 · similarity 0.85)
- Training Deep Neural Networks with Virtual Smoothing Classes (AAAI 2025 · similarity 0.85)
- NoiseGPT: Label Noise Detection and Rectification through Probability Curvature (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.