Research map / Label-efficient and robust learning / Noisy labels
Noisy Label Learning with Instance-Dependent Outliers: Identifiability via Crowd Wisdom
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that single-annotator labels cannot identify instance-dependent outliers in confusion-matrix noisy label learning, and proposes COINNet, a crowdsourcing loss with an l2/lp column-sparsity outlier penalty and volume regularization, with identifiability guarantees.
Problem
Learning a ground-truth classifier under label noise modeled by confusion matrices when the confusion matrix is occasionally instance-dependent, which introduces outliers; with a single annotator these outliers are not identifiable.
Approach
Use multiple annotators so that nominal labels lie in a low-dimensional subspace (range of stacked annotator confusion matrices) while instance-dependent outliers are likely outside it; combine a coupled cross-entropy loss with a differentiable l2/lp column-sparsity penalty on outlier terms and a volume regularizer, giving one-stage end-to-end training.
Key results
- On CIFAR-10 with machine annotators (high noise), COINNet reaches 71.22% accuracy vs 67.22% for the second-best GeoCrowdNet(F).
- On CIFAR-10N, COINNet achieves 92.09% vs 90.11% for MaxMIG; on LabelMe 87.60% vs 87.21% for GeoCrowdNet(F).
- On ImageNet-15N (real AMT annotations), COINNet achieves 93.71% vs 81.13% for MaxMIG.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · CIFAR-10 high noise | classification accuracy: 67.22 ± 0.27 -> 71.22 ± 0.72 | Helped |
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · CIFAR-10 medium noise | classification accuracy: 71.91 ± 0.50 -> 73.31 ± 0.09 | Helped |
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · CIFAR-10 low noise | classification accuracy: 83.16 ± 0.53 -> 84.14 ± 0.38 | Helped |
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · STL-10 high noise | classification accuracy: 67.92 ± 0.49 -> 70.12 ± 0.48 | Helped |
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · STL-10 medium noise | classification accuracy: 71.75 ± 0.20 -> 73.11 ± 0.37 | Helped |
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · STL-10 low noise | classification accuracy: 74.32 ± 0.76 -> 76.39 ± 0.58 | Helped |
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · CIFAR-10N | classification accuracy: 90.11 ± 0.09 -> 92.09 ± 0.47 | Helped |
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · LabelMe | classification accuracy: 87.21 ± 0.39 -> 87.60 ± 0.54 | Helped |
| Use COINNet instead of the strongest reported baseline | Noisy-label classifier · ImageNet-15N | classification accuracy: 81.13 ± 1.42 -> 93.71 ± 3.26 | Helped |
| Use multiple annotators with the column-sparsity outlier constraint rather than one annotator | Confusion-matrix noisy-label learning · Identifiability under instance-dependent outliers | outlier detection and classifier identifiability | Helped |
Surprising findings
- Using only one annotator, a sparsity constraint on outliers cannot identify outliers: a trivial solution with an identity confusion matrix and zero outliers attains the optimum for universal function approximators.
- Instance-dependent baselines (MEIDTM, PTD, BLTM) degrade notably under high noise, which the authors attribute to error accumulation in multi-stage training.
Limitations
- Assumes outliers do not occur overwhelmingly; all-instance-dependent noise is not handled.
- Outlier structure is ignored (treated agnostically), so the outlier generating function is not learned.
- Identifiability guarantees rely on assumptions such as class specialists and anchor points, or the SSC condition, and multiple annotators.
Experimental setup
Datasets: CIFAR-10 STL-10 CIFAR-10N LabelMe ImageNet-15N
Benchmarks: CIFAR-10 STL-10 CIFAR-10N LabelMe ImageNet-15N
Metrics: classification accuracy
Baselines: GeoCrowdNet(F) GeoCrowdNet(W) TraceReg CrowdLayer MaxMIG PTD BLTM MEIDTM VolMinNet Reweight GCE
Builds on: coupled cross-entropy minimization volume minimization (VolMinNet) GeoCrowdNet nonnegative matrix factorization identifiability
Closest papers by meaning
- Unbiased Multi-Label Learning from Crowdsourced Annotations (ICML 2024 · similarity 0.88)
- Coupled Confusion Correction: Learning from Crowds with Sparse Annotations (AAAI 2024 · similarity 0.88)
- Learning from Noisy Labels via Conditional Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.88)
- Learning Causal Transition Matrix for Instance-dependent Label Noise (AAAI 2025 · similarity 0.87)
- Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection (CVPR 2024 · similarity 0.86)
- Learning the Latent Causal Structure for Modeling Label Noise (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.