Research map / Label-efficient and robust learning / Noisy labels
Sample selection and co-training
Separates clean from noisy samples using small-loss selection, co-teaching and joint optimization.
86 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Noisy labels, keyword cluster “noisy labels · robust · clean”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
24 papers in 2023–24 and 7 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 19.8% to 5.7%.
Open questions the data points to
- Papers report Co-teaching+ on CIFAR-100 Symmetry-20% (test accuracy (%)) anywhere from 49.27 to 56.15: comparisons against it depend on whose number you take.
- Papers report Crust on CIFAR-100, Sym 50 (test accuracy) anywhere from 55.5 to 62.3: comparisons against it depend on whose number you take.
No longer reported by papers here: Clothing1M (test accuracy, last in 2023); Clothing1M test (accuracy, last in 2024); ANIMAL-10N (test accuracy, last in 2024); CIFAR-10N worst (test accuracy (%), last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Robust Loss Functions under Label Noise for Deep Neural Networks (AAAI 2017 · 930 citations)
- 2018 · Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels (NeurIPS 2018 · 1,467 citations)
- 2019 · Probabilistic End-To-End Noise Correction for Learning With Noisy Labels (CVPR 2019 · 454 citations)
- 2020 · Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization (CVPR 2020 · 600 citations)
- 2021 · Jo-SRC: A Contrastive Approach for Combating Noisy Labels (CVPR 2021 · 183 citations)
- 2022 · Selective-Supervised Contrastive Learning With Noisy Labels (CVPR 2022 · 217 citations)
- 2023 · DISC: Learning From Noisy Labels via Dynamic Instance-Specific Selection and Correction (CVPR 2023 · 121 citations)
- 2024 · L2B: Learning to Bootstrap Robust Models for Combating Label Noise (CVPR 2024 · 20 citations)
- 2025 · Directional Label Diffusion Model for Learning from Noisy Labels (CVPR 2025 · 4 citations)
- 2026 · Trapped by simplicity: When Transformers fail to learn from noisy features (ICLR 2026 · 0 citations)
Earliest papers here
- Robust Loss Functions under Label Noise for Deep Neural Networks (AAAI 2017 · 930 citations)
- Training deep neural-networks using a noise adaptation layer (ICLR 2017 · 511 citations)
- Decoupling "when to update" from "how to update" (NeurIPS 2017 · 201 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
Clothing1M · accuracy (higher is better)
12 papers, 2019–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 73.49 | Probabilistic End-To-End Noise Correction for Learning With Noisy Labels (CVPR 2019 · 454 citations) |
| 2020 | 74.3 | Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels (ICML 2020 · 116 citations) |
| 2021 | 75.93 | Jo-SRC: A Contrastive Approach for Combating Noisy Labels (CVPR 2021 · 183 citations) |
Clothing1M · test accuracy (higher is better)
9 papers, 2019–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 72.46 | L_DMI: A Novel Information-theoretic Loss Function for Training Deep Nets Robust to Label Noise (NeurIPS 2019 · 120 citations) |
| 2020 | 74.81 | Early-Learning Regularization Prevents Memorization of Noisy Labels (NeurIPS 2020 · 257 citations) |
| 2021 | 74.99 | Understanding and Improving Early Stopping for Learning with Noisy Labels (NeurIPS 2021 · 53 citations) |
Clothing1M test · accuracy (higher is better)
9 papers, 2018–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 72.23 | Joint Optimization Framework for Learning With Noisy Labels (CVPR 2018 · 728 citations) |
| 2019 | 73.72 | Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (NeurIPS 2019 · 398 citations) |
| 2020 | 74.1 | A Topological Filter for Learning with Label Noise (NeurIPS 2020 · 12 citations) |
| 2023 | 75.63 | Learning With Noisy Labels via Self-Supervised Adversarial Noisy Masking (CVPR 2023 · 25 citations) |
ANIMAL-10N · accuracy (higher is better)
5 papers, 2022–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 84.18 | Noise Attention Learning: Enhancing Noise Robustness by Gradient Scaling (NeurIPS 2022 · 2 citations) |
| 2023 | 89.3 | Learning With Noisy Labels via Self-Supervised Adversarial Noisy Masking (CVPR 2023 · 25 citations) |
| 2025 | 89.4 | Directional Label Diffusion Model for Learning from Noisy Labels (CVPR 2025 · 4 citations) |
ANIMAL-10N · test accuracy (higher is better)
4 papers, 2022–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 86.71 | On Learning Contrastive Representations for Learning With Noisy Labels (CVPR 2022 · 76 citations) |
| 2023 | 96.37 | Leveraging Unlabeled Data to Track Memorization (ICLR 2023 · 0 citations) |
CIFAR-10N worst · test accuracy (%) (higher is better)
4 papers, 2023–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 88.74 | Mitigating Memorization of Noisy Labels via Regularization between Representations (ICLR 2023 · 8 citations) |
| 2024 | 93.82 | Dirichlet-Based Prediction Calibration for Learning with Noisy Labels (AAAI 2024 · 17 citations) |
What has been tried
78 papers here report 863 measured changes (a component added, removed or swapped, compared with a baseline): helped 71%, no clear effect 8%, hurt 17%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Set α = 0, removing the observed-label loss | CIFAR-100, 20% and 40% symmetric noise · test accuracy (%) | 71.8 -> 55.7; 67.3 -> 47.1 | Hurt | L2B: Learning to Bootstrap Robust Models for Combating Label Noise (CVPR 2024 · 20 citations) |
| Use reverse k-NN with kst 20 | CIFAR10-IDN noise rate 0.20 · accuracy | Helped | Learning with Structural Labels for Learning with Noisy Labels (CVPR 2024 · 20 citations) | |
| Add the EDL loss | CIFAR-10 symmetric 50% · last-epoch test accuracy | 94.7 -> 95.2 | Helped | Dirichlet-Based Prediction Calibration for Learning with Noisy Labels (AAAI 2024 · 17 citations) |
| Add label pre-correction | CIFAR-10 and CIFAR-100 noisy-label experiments · label accuracy | Helped | Directional Label Diffusion Model for Learning from Noisy Labels (CVPR 2025 · 4 citations) | |
| Add both mimicry losses | CIFAR-100, Sym 80% · accuracy | 68.2 -> 70.3 | Helped | Combating Semantic Contamination in Learning with Label Noise (AAAI 2025 · 3 citations) |
| Add logit bias ε=0.5 to MAE | Cifar-10, symmetric noise 0% · accuracy | 84.90 ± 1.86 -> 91.27 ± 0.13 | Helped | Enhancing Noise-Robust Losses for Large-Scale Noisy Data Learning (AAAI 2025 · 1 citation) |
| Use ensemble IOFM scores across epochs instead of a single final-epoch IOFM score | Imbalanced CIFAR100 and heavily noisy CIFAR10/CIFAR100 · clean/noisy classification AUC | Helped | IOFM: Using the Interpolation Technique on the Over-Fitted Models to Identify Clean-Annotated Samples (AAAI 2024 · 1 citation) | |
| Add a gating network | CIFAR-10H, CIFAR-10N, CIFAR-100N, ImageNet-PI low-noise and high-noise · test accuracy | 61.5 -> 71.3; 84.5 -> 84.9; 59.0 -> 64.2; 67.9 -> 71.6; 47.8 -> 62.1 | Helped | Pi-DUAL: Using privileged information to distinguish clean from noisy labels (ICML 2024 · 0 citations) |
| Increase m from 0 to 1e2 | CIFAR-100 symmetric noise 0.4 · accuracy | 39.72 ±0.67 -> 48.39 ±0.53 | Helped | $\epsilon$-Softmax: Approximating One-Hot Vectors for Mitigating Label Noise (NeurIPS 2024 · 0 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Co-teaching+ | CIFAR-100 Symmetry-20% | test accuracy (%) | 49.27 – 56.15 | 2 papers, e.g. CVPR 2020 and ICLR 2020 |
| Crust | CIFAR-100, Sym 50 | test accuracy | 55.5 – 62.3 | 2 papers, e.g. ICML 2022 and NeurIPS 2021 |
| Co-teaching | ANIMAL-10N | test accuracy | 85.73 – 92.2 | 2 papers, e.g. CVPR 2022 and NeurIPS 2024 |
What reviewers will expect
Datasets and baselines named in the 5 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 4 of 5 papers
- CIFAR-100 · 3 of 5 papers
- WebVision · 3 of 5 papers
- Animal-10N · 2 of 5 papers
Baselines
- Co-teaching · 2 of 5 papers
- DivideMix · 2 of 5 papers
- DISC · 2 of 5 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 31 papers; from OpenAlex and the papers' affiliations).
- Nanyang Technological University · 3 papers
- Tencent · 3 papers
- The University of Hong Kong · 2 papers
- Shanghai Jiao Tong University · 2 papers
- University of California, Santa Cruz · 2 papers
- Google · 2 papers
- EPFL · 2 papers
- RIKEN · 2 papers
Use it
Download BibTeX of 35 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Noise-aware loss and pairwise methods (66 papers)
- Contrastive learning on noisy data (55 papers)
- Noisy cross-modal hashing and multi-label (49 papers)
Elsewhere in the topic
- Instance-dependent noise modeling (106 papers)
- Loss correction and noise robustness theory (77 papers)
- Crowdsourced label aggregation (23 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).