Research map / Learning theory and algorithms / Prediction and decision losses
Learning from Noisy Labels via Conditional Distributionally Robust Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates learning from crowdsourced noisy labels as conditional distributionally robust optimization (CDRO) with a Wasserstein ambiguity set, derives a closed-form robust pseudo-labeling rule and closed-form robust risk, and trains AdaptCDRP with an adaptive Lagrange multiplier.
Problem
Learning accurate classifiers from noisy crowdsourced labels when the estimated true-label posterior (from noise transition estimates and priors) is misspecified, which degrades performance especially at high noise.
Approach
Minimize worst-case risk over a Wasserstein ambiguity set around an estimated true-label posterior; the dual yields a per-instance analytical solution expressed as a likelihood-ratio test that produces robust pseudo-labels (only for high-confidence instances), a pseudo-empirical reference distribution, and a closed-form empirical robust risk with a one-step update for the Lagrange multiplier.
Key results
- On CIFAR-10 with IDN-HIGH noise (R=5, one annotation per instance), AdaptCDRP reaches 86.62±0.45% vs 78.67% for the next best (CoDis).
- On CIFAR-100 with IDN-HIGH noise, AdaptCDRP reaches 54.24±0.99% vs 49.00% for LogitClip.
- On real-world datasets AdaptCDRP achieves the best reported accuracy on CIFAR-10N (88.25%), CIFAR-100N (53.42%), LabelMe (83.36%) and Animal-10N (83.08%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AdaptCDRP instead of LogitClip | CIFAR-10 IDN-LOW · CIFAR-10 IDN-LOW | accuracy: 85.39 ± 0.35 -> 88.09 ± 0.37 | Helped |
| Use AdaptCDRP instead of CoDis | CIFAR-10 IDN-MID · CIFAR-10 IDN-MID | accuracy: 82.02 ± 0.41 -> 87.37 ± 0.29 | Helped |
| Use AdaptCDRP instead of Co-teaching | CIFAR-10 IDN-HIGH · CIFAR-10 IDN-HIGH | accuracy: 81.30 ± 0.82 -> 86.62 ± 0.45 | Helped |
| Use AdaptCDRP instead of CoDis | CIFAR-100 IDN-LOW · CIFAR-100 IDN-LOW | accuracy: 58.66 ± 0.44 -> 60.20 ± 0.15 | Helped |
| Use AdaptCDRP instead of LogitClip | CIFAR-100 IDN-MID · CIFAR-100 IDN-MID | accuracy: 53.14 ± 0.37 -> 56.65 ± 1.03 | Helped |
| Use AdaptCDRP instead of LogitClip | CIFAR-100 IDN-HIGH · CIFAR-100 IDN-HIGH | accuracy: 49.00 ± 0.35 -> 54.24 ± 0.99 | Helped |
| Use AdaptCDRP instead of CoDis | CIFAR-10N · CIFAR-10N | accuracy: 87.23 ± 0.45 -> 88.25 ± 0.34 | Helped |
| Use AdaptCDRP instead of CoDis | CIFAR-100N · CIFAR-100N | accuracy: 52.66 ± 0.44 -> 53.42 ± 0.64 | Helped |
| Use AdaptCDRP instead of Max-MIG | LabelMe · LabelMe | accuracy: 83.25 ± 0.26 -> 83.36 ± 0.68 | Helped |
| Use AdaptCDRP instead of Co-teaching | Animal-10N · Animal-10N | accuracy: 81.96 ± 0.48 -> 83.08 ± 0.39 | Helped |
Surprising findings
- Gains are largest under high noise: the advantage over baselines grows as the annotator error rate increases from about 20% to 50%.
Limitations
- Noise transition probabilities are estimated by simple frequency counting, which may be imprecise with sparse annotations.
- Paper states it does not focus on precisely estimating the noise transition matrix or true label posterior.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CIFAR-10N CIFAR-100N LabelMe Animal-10N
Benchmarks: CIFAR-10 CIFAR-100 CIFAR-10N CIFAR-100N LabelMe Animal-10N
Metrics: test accuracy
Baselines: CE (MV) CE (EM) Co-teaching Co-teaching+ CoDis LogitClip DoctorNet MBEM CrowdLayer TraceReg Max-MIG CoNAL
Builds on: distributionally robust optimization co-teaching CoDis
Closest papers by meaning
- Unbiased Multi-Label Learning from Crowdsourced Annotations (ICML 2024 · similarity 0.88)
- Label Noise: Ignorance Is Bliss (NeurIPS 2024 · similarity 0.88)
- Noisy Label Learning with Instance-Dependent Outliers: Identifiability via Crowd Wisdom (NeurIPS 2024 · similarity 0.88)
- Uncertainty Herding: One Active Learning Method for All Label Budgets (ICLR 2025 · similarity 0.87)
- Semi-Supervised Regression with Heteroscedastic Pseudo-Labels (NeurIPS 2025 · similarity 0.87)
- Understanding Domain Generalization: A Noise Robustness Perspective (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.