Research map / Label-efficient and robust learning / Noisy labels
Regroup Median Loss for Combating Label Noise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Regroup Median Loss (RML), a robust per-sample loss estimator that reweights same-label sample selection and combines group mean losses with a median, plus a semi-supervised variant using MixMatch, for learning with noisy labels.
Problem
Deep models trained on datasets with noisy labels overfit to incorrect annotations; small-loss sample selection still admits noisy samples that look clean, so noisy samples are discarded or misused.
Approach
Process per-sample CE losses (l*(l+eps)) to sharpen selection probabilities toward clean samples, randomly draw same-label samples, split them into groups, and estimate each sample's loss as the median of group mean losses plus its own loss (a Median-of-Means style robust estimator); a momentum model and a agreement-based split enable a MixMatch-based semi-supervised extension.
Key results
- On CIFAR-100 with noisy labels, RML improves test accuracy by about 6% over prior methods without semi-supervised training (e.g., 67.05 vs 50.22 for Joint-Optim at symmetric-0.5).
- On WebVision, RML reaches 73.32% vs 65.24% for Iterative-CV, and the semi-supervised RML-Semi reaches 81.34% vs 79.96% for Sel-CL.
- On Clothing1M, RML-Semi achieves 75.14% vs 74.99% for PES, a 0.15% gain over the best prior semi-supervised method.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add loss processing to RML (ablation) | RML · CIFAR-10 and CIFAR-100 with instance-dependent, symmetric, and pairflip noise | test accuracy (%): CIFAR-10: 88.47 $\pm$ 0.37, 86.73 $\pm$ 0.33, 87.75 $\pm$ 0.71 -> 90.73 $\pm$ 0.44, 89.13 $\pm$ 0.21, 90.14 $\pm$ 0.34; | Helped |
| Add the regroup median operation instead of computing mean selected-sample loss (ablation) | RML · CIFAR-10 and CIFAR-100 with instance-dependent, symmetric, and pairflip noise | test accuracy (%): CIFAR-10: 85.32 $\pm$ 0.54, 83.61 $\pm$ 0.43, 85.77 $\pm$ 0.25 -> 90.73 $\pm$ 0.44, 89.13 $\pm$ 0.21, 90.14 $\pm$ 0.34; | Helped |
| Vary median window size n (ablation) | RML · CIFAR-10 instance, symmetric, and pairflip noise | test accuracy (%): Instance: 87.41$\pm$0.86 (n=2), 90.29$\pm$0.24 (n=4), 90.73$\pm$0.44 (n=6), 90.22$\pm$0.73 (n=8), 89.71$\pm$0.23 (n=10); | Mixed |
| Add the semi-supervised strategy to RML | RML · CIFAR-10 and CIFAR-100 with symmetric, instance, and pairflip noise | test accuracy (%): null | Helped |
| Use RML-Semi instead of the strongest reported baseline | RML-Semi · Clothing1M, WebVision, and ILSVRC12 | test accuracy (%): Clothing1M: 74.99 -> 75.14; WebVision: 79.96 -> 81.34; ILSVRC12: 76.84 -> 77.38 | Helped |
Surprising findings
- Ablation shows that larger window size n slows convergence because the estimated loss becomes smaller, so n=6 was chosen as a trade-off.
Limitations
- Requires several hyperparameters (k, n, epsilon) that vary with noise rate and dataset, e.g., k changes per setting.
- Loss-correction rule and per-epoch loss update are heuristic additions beyond the theoretical estimator.
- Theoretical robustness bound relies on the assumption that less than half of subsets are contaminated, which is not verified in practice.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Clothing1M WebVision ILSVRC12
Benchmarks: CIFAR-10 CIFAR-100 Clothing1M WebVision ILSVRC12
Metrics: average test accuracy (%)
Baselines: CE Co-teaching Forward Joint-Optim T-revision DMI PES MixUp DivideMix ELR+ UNICON Sel-CL
Builds on: Co-teaching DivideMix MixMatch Median-of-Means
Closest papers by meaning
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.87)
- Mitigating Label Noise through Data Ambiguation (AAAI 2024 · similarity 0.86)
- Enhancing Robustness of Last Layer Two-Stage Fair Model Corrections (NeurIPS 2024 · similarity 0.86)
- NoiseGPT: Label Noise Detection and Rectification through Probability Curvature (NeurIPS 2024 · similarity 0.85)
- Learning from Noisy Labels via Self-Taught On-the-Fly Meta Loss Rescaling (AAAI 2025 · similarity 0.85)
- Handling Label Noise via Instance-Level Difficulty Modeling and Dynamic Optimization (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.