Research map / Label-efficient and robust learning / Noisy labels
Learning From Noisy Singly-labeled Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MBEM to estimate worker quality and train from noisy labels, even with one annotation per example | ImageNet with simulated workers · ImageNet-1K | generalization error | Helped |
| Use MBEM to estimate worker quality and train from noisy labels, even with one annotation per example | MBEM · CIFAR-10, hammer-spammer workers, redundancy 1 | generalization error | Helped |
| Use MBEM with real noisy worker annotations | MS-COCO · MS-COCO test | F1 score | Helped |
| Use MBEM rather than standard EM when the annotation budget is fixed and worker quality is above the threshold | MBEM · CIFAR-10, fixed annotation budget | generalization error | Helped |
| Allocate a fixed annotation budget to many singly labeled examples rather than fewer multiply labeled examples | MBEM · MS-COCO | F1 score | Helped |
| Weight training loss using posterior probabilities of the true labels instead of using majority vote labels (ablation) | weighted-MV versus MV · CIFAR-10, hammer-spammer workers, varying redundancy | generalization error | Helped |
| Weight training loss using posterior probabilities of the true labels instead of using majority vote labels (ablation) | weighted-EM versus EM · CIFAR-10, hammer-spammer workers, varying redundancy | generalization error | Helped |
| Add model-prediction bootstrapping to worker-quality estimation (ablation) | MBEM versus weighted-EM · CIFAR-10, small redundancy | generalization error | Helped |
| Increase redundancy while holding the total annotation budget fixed (ablation) | MBEM · CIFAR-10, redundancy 1 to 5 | generalization error | Hurt |
| Increase redundancy while holding the total annotation budget fixed (ablation) | weighted-EM · CIFAR-10, redundancy 1 to 5 | generalization error | Helped |
| Use standard EM rather than MBEM (ablation) | CIFAR-10 · CIFAR-10, large redundancy | generalization error | No clear effect |
| Increase redundancy while holding total annotation budget fixed (ablation) | MBEM · ImageNet-1K | generalization error | Hurt |
Closest papers by meaning
- Label Correction of Crowdsourced Noisy Annotations with an Instance-Dependent Noise Transition Model (NeurIPS 2023 · similarity 0.91)
- Coupled Confusion Correction: Learning from Crowds with Sparse Annotations (AAAI 2024 · similarity 0.89)
- Unbiased Multi-Label Learning from Crowdsourced Annotations (ICML 2024 · similarity 0.88)
- Noisy Label Learning with Instance-Dependent Outliers: Identifiability via Crowd Wisdom (NeurIPS 2024 · similarity 0.88)
- Learning from Noisy Labels via Conditional Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.88)
- Collaborative Refining for Learning from Inaccurate Labels (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.