Research map / Label-efficient and robust learning / Noisy labels
Does Label Smoothing Help Deep Partial Label Learning?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows theoretically and empirically that label smoothing helps deep partial label learning, deriving bounds and an optimal smoothing rate tied to a generalized ambiguity degree, and proposes the LS-PLL algorithm.
Problem
Deep partial label learning (PLL) classifiers are degraded by noisy false-positive candidate labels, especially competitive (instance-dependent) noise, as training proceeds, and lack theory linking label noise or ambiguity degree to classification performance.
Approach
Defines a smoothed partial label loss that mixes the identified ground-truth label with a uniform distribution over the candidate set, proves lower and upper bounds relating it to the clean-label smoothed risk, derives an optimal smoothing rate from a generalized ambiguity degree, and pairs this with an alternating label-identification algorithm (LS-PLL).
Key results
- On CIFAR-10 (ResNet-18), LS-PLL reaches 82.97% accuracy at Avg. #CL=3 vs 63.39% without label smoothing; gains grow with ambiguity.
- On CIFAR-100 (ResNet-56) at Avg. #CL=7, LS-PLL achieves 57.90% vs 34.15% without label smoothing.
- Empirical best smoothing rates (0.3, 0.3, 0.7 on CIFAR-10) approximate theoretical optimal rates (0.24, 0.26, 0.68) computed from the generalized ambiguity degree.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add label smoothing to LeNet-5 | Deep PLL · Fashion-MNIST, Avg.#CL=3/4/5 | accuracy: 87.80 -> 89.99; 83.36 -> 84.32; 81.23 -> 84.06 | Helped |
| Add label smoothing to LeNet-5 | Deep PLL · Kuzushiji-MNIST, Avg.#CL=3/4/5 | accuracy: 85.28 -> 90.64; 76.31 -> 87.85; 67.25 -> 76.10 | Helped |
| Add label smoothing to ResNet-18 | Deep PLL · CIFAR-10, Avg.#CL=3/4/5 | accuracy: 63.39 -> 82.97; 51.47 -> 77.68; 38.96 -> 67.23 | Helped |
| Add label smoothing to ResNet-56 | Deep PLL · CIFAR-100, Avg.#CL=7/9/11 | accuracy: 34.15 -> 57.90; 33.60 -> 47.50; 21.00 -> 35.50 | Helped |
| Vary the smoothing rate across 0.1, 0.3, 0.5, 0.7, and 0.9 (ablation) | LS-PLL on Fashion-MNIST · Fashion-MNIST, Avg.#CL=3/4/5 | accuracy: 84.39, 84.72, 84.73, 84.80, 89.99; 72.96, 66.89, 76.42, 81.48, 84.32; 52.55, 55.87, 47.56, 47.95, 84.06 | Mixed |
| Vary the smoothing rate across 0.1, 0.3, 0.5, 0.7, and 0.9 (ablation) | LS-PLL on Kuzushiji-MNIST · Kuzushiji-MNIST, Avg.#CL=3/4/5 | accuracy: 81.11, 88.22, 89.34, 89.35, 90.64; 47.99, 53.39, 55.17, 77.77, 87.85; 36.10, 33.68, 50.50, 34.37, 76.10 | Mixed |
| Vary the smoothing rate across 0.1, 0.3, 0.5, 0.7, and 0.9 (ablation) | LS-PLL on CIFAR-10 · CIFAR-10, Avg.#CL=3/4/5 | accuracy: 82.62, 82.97, 82.06, 82.19, 81.31; 64.20, 77.68, 76.95, 77.24, 75.99; 48.99, 53.09, 62.31, 62.92, 62.38 | Mixed |
| Vary the smoothing rate across 0.1, 0.3, 0.5, 0.7, and 0.9 (ablation) | LS-PLL on CIFAR-100 · CIFAR-100, Avg.#CL=7/9/11 | accuracy: 37.62, 38.00, 43.50, 48.67, 57.90; 27.40, 20.00, 25.80, 38.80, 47.50; 16.40, 22.70, 22.10, 25.10, 33.10 | Mixed |
Surprising findings
- Most datasets (Fashion-MNIST, Kuzushiji-MNIST, CIFAR-100) prefer very high smoothing rates (r=0.9), while CIFAR-10 prefers moderate rates.
Limitations
- Experiments limited to small image benchmarks and simple architectures (LeNet-5, ResNet-18/56).
- Candidate label sets are synthetically generated rather than from real annotators.
- Ambiguity degree and optimal smoothing rate require knowledge of the noise structure, which is not available in practice.
Experimental setup
Datasets: Fashion-MNIST Kuzushiji-MNIST CIFAR-10 CIFAR-100
Benchmarks: Fashion-MNIST Kuzushiji-MNIST CIFAR-10 CIFAR-100
Metrics: test accuracy pre-logit cluster visualization
Baselines: LS-PLL without label smoothing (w/o LS)
Builds on: label smoothing (Szegedy et al., 2016) PRODEN Yan & Guo (2023) competitive noise PLL
Closest papers by meaning
- Realistic Evaluation of Deep Partial-Label Learning Algorithms (ICLR 2025 · similarity 0.89)
- What Makes Partial-Label Learning Algorithms Effective? (NeurIPS 2024 · similarity 0.88)
- Conformal Prediction for Partial Label Learning (AAAI 2025 · similarity 0.88)
- Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning (AAAI 2024 · similarity 0.87)
- Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It (ICLR 2025 · similarity 0.86)
- For Better or For Worse? Learning Minimum Variance Features With Label Augmentation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.