Research map / Label-efficient and robust learning / Noisy labels
L2B: Learning to Bootstrap Robust Models for Combating Label Noise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Learning to Bootstrap (L2B), a meta-learned objective that jointly reweights observed and pseudo-labels per sample (weights need not sum to one), implicitly relabeling noisy data, and integrates with DivideMix, UniCon, and C2D.
Problem
Training deep networks with noisy labels causes overfitting; existing bootstrapping losses use fixed or sum-to-one mixing of observed and pseudo-labels and treat all samples equally, and meta-reweighting methods need clean validation sets.
Approach
Replace the convex label mixture of bootstrapping with separate per-sample weights on the observed-label loss and the pseudo-label loss (not constrained to sum to one), meta-learned via one-step validation gradients, so the method performs implicit relabeling while also reweighting instances; an online GMM-based meta set can replace the clean validation set.
Key results
- On CIFAR-100 with 50% symmetric noise, L2B reaches 64.5% vs 55.6% for L2RW and 44.4% for cross-entropy (+8.9% over L2RW).
- L2B-C2D achieves 94.4% on CIFAR-10 and 60.7% on CIFAR-100 at 90% symmetric noise, the best among compared methods in Table 2.
- On Clothing 1M, L2B-Naive reaches 77.5 ± 0.2% accuracy, above the listed competitors (best prior 76.0%, MSLG).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use L2B instead of L2RW | PreActResNet-18 · CIFAR-10, 20%–50% symmetric noise | test accuracy (%): 90.6 -> 92.2; 89.0 -> 90.7; 86.6 -> 89.9; 85.3 -> 88.5 | Helped |
| Use L2B instead of L2RW | PreActResNet-18 · CIFAR-100, 20%–50% symmetric noise | test accuracy (%): 67.8 -> 71.8; 63.8 -> 69.5; 59.7 -> 67.3; 55.6 -> 64.5 | Helped |
| Use L2B instead of the strongest listed baseline | ResNet-50 · ISIC2019, 20%–50% symmetric noise | test accuracy (%): 80.8 -> 81.1; 77.7 -> 80.2; 76.3 -> 78.6; 74.8 -> 76.8 | Helped |
| Add L2B to C2D | CIFAR-10 · 40% asymmetric noise | test accuracy: 93.4 -> 94.0 | Helped |
| Use L2B-Naive instead of MSLG | Clothing 1M · Clothing 1M | accuracy (%): 76.0 -> 77.5 ± 0.2 | Helped |
| Add L2B to the UNet++ segmentation baseline | PROMISE12 · PROMISE12, L1–L3 noise | Dice (%): 59.77 -> 77.70; 73.74 -> 80.83; 80.03 -> 82.01 | Helped |
| Set α = 0, removing the observed-label loss (ablation) | L2B · CIFAR-100, 20% and 40% symmetric noise | test accuracy (%): 71.8 -> 55.7; 67.3 -> 47.1 | Hurt |
| Set β = 0, removing the pseudo-label loss (ablation) | L2B · CIFAR-100, 20% and 40% symmetric noise | test accuracy (%): 71.8 -> 63.2; 67.3 -> 57.5 | Hurt |
| Constrain α + β = 1 (ablation) | L2B · CIFAR-100, 20% and 40% symmetric noise | test accuracy (%): 64.8 vs. 71.8; 59.1 vs. 67.3 | Hurt |
Surprising findings
- Constraining the two weights to sum to one (α+β=1) yields lower accuracy than unconstrained weights (59.1% vs 67.3% at 40% noise on CIFAR-100).
- Using only pseudo-label loss (α=0) performs worse than the cross-entropy baseline on CIFAR-100 at 20% and 40% noise.
Limitations
- Relies on a small clean validation set for meta-learning, though an online GMM-based meta set is used as an alternative.
- Performance on high-noise settings with DivideMix is still limited (e.g., 35.8% on CIFAR-100 at 90%).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ISIC2019 Clothing 1M PROMISE12
Benchmarks: CIFAR-10 CIFAR-100 ISIC2019 Clothing 1M PROMISE12
Metrics: test accuracy Dice Hausdorff distance average surface distance
Baselines: Cross-Entropy Bootstrap L2RW Co-teaching+ Mixup PENCIL DivideMix UniCon C2D MSLC TCL NL reweighting
Builds on: Bootstrap loss L2RW DivideMix UniCon C2D
Closest papers by meaning
- Unsupervised Label Noise Modeling and Loss Correction (ICML 2019 · similarity 0.88)
- Directional Label Diffusion Model for Learning from Noisy Labels (CVPR 2025 · similarity 0.87)
- Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagation (ICLR 2024 · similarity 0.87)
- NoiseGPT: Label Noise Detection and Rectification through Probability Curvature (NeurIPS 2024 · similarity 0.87)
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.86)
- MaxSup: Overcoming Representation Collapse in Label Smoothing (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.