Research map / Label-efficient and robust learning / Noisy labels
Combating Semantic Contamination in Learning with Label Noise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies "Semantic Contamination" in label refurbishment methods for learning with noisy labels and proposes Collaborative Cross Learning, which combines semi-supervised cross-view and cross-model contrastive learning on refurbished labels to learn consistent semantic structure.
Problem
Label refurbishment methods for learning with noisy labels can learn corrupted semantic associations between classes (Semantic Contamination), causing imbalanced and inconsistent embeddings across views and models.
Approach
Decouple refurbishment into predicted-class guidance and semantic-concept terms; replace logit-level semantic smoothing with embedding-level augmentation-wise contrastive learning and view/model mimicry, and use contrastive cross-model learning on refurbished labels (CCLRL) to maximize mutual information between the two peer models.
Key results
- On CIFAR-100 with 80% symmetric noise, the method reaches 70.3% vs. 66.2% for RoLR and 67.2% for RankMatch.
- On mini-WebVision, top-1 accuracy is 82.3% vs. 81.8% for RoLR, and ILSVRC12 top-1 is 78.2%.
- On Animal-10N, the method achieves 89.7% vs. 88.5% for RoLR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Collaborative Cross Learning rather than RoLR | Label-noise learning · CIFAR-100, Sym 80% | accuracy: 66.2% to 70.3% | Helped |
| Add Cross-view and Cross-model learning losses to DivideMix | DivideMix · CIFAR-100, Sym 80% | accuracy: 59.6 -> 68.4 | Helped |
| Add Cross-model learning loss to DivideMix | DivideMix · CIFAR-100, Sym 80% | accuracy: 59.6 -> 64.2 | Helped |
| Add Cross-view learning loss to DivideMix | DivideMix · CIFAR-100, Sym 80% | accuracy: 59.6 -> 62.3 | Helped |
| Add both mimicry losses (ablation) | Collaborative Cross Learning · CIFAR-100, Sym 80% | accuracy: 68.2 -> 70.3 | Helped |
| Add both mimicry losses (ablation) | Collaborative Cross Learning · CIFAR-10, Sym 50% | accuracy: 95.7 -> 96.5 | Helped |
| Add the confidence threshold in the predicted-guidance loss (ablation) | Collaborative Cross Learning · CIFAR-100, Sym 80% | accuracy: 67.9 -> 70.3 | Helped |
| Use refurbished labels instead of vanilla contrastive learning (ablation) | Collaborative Cross Learning · CIFAR-100, Sym 80% | accuracy: 68.4 -> 70.3 | Helped |
| Add Cross-view learning loss (ablation) | Collaborative Cross Learning · CIFAR-100, Sym 80% | accuracy: 68.2 -> 70.3 | Helped |
| Add Cross-model learning loss (ablation) | Collaborative Cross Learning · CIFAR-100, Sym 80% | accuracy: 66.2 -> 70.3 | Helped |
| Use one weak and one strong augmentation instead of the same augmentation for both views (ablation) | Collaborative Cross Learning · CIFAR-100, Sym 80% | accuracy | Mixed |
| Set temperature too high or too low (ablation) | Collaborative Cross Learning · CIFAR-10 and CIFAR-100, Sym 80% | accuracy | Hurt |
Surprising findings
- Semi-supervised methods such as DivideMix still suffer from Semantic Contamination, and adding the proposed losses to DivideMix improves its accuracy (e.g., CIFAR-100 80% noise 59.6 to 68.4).
Limitations
- Theoretical analysis of the generalization and foundation is left for future work
- Gains on CIFAR-10 at 80% symmetric noise are small (94.6 vs. 94.2 for RoLR; some baselines close)
- Relies on two-model peer training with warm-up and GMM confidence estimation, adding design complexity
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CIFAR-10N CIFAR-100N Animal-10N WebVision ILSVRC12
Benchmarks: CIFAR-10 CIFAR-100 CIFAR-10N CIFAR-100N Animal-10N WebVision ILSVRC12
Metrics: test accuracy top-1 accuracy top-5 accuracy LCA cosine similarity consistency Wasserstein distance
Baselines: Co-teaching SCE JoCoR DivideMix Co-learning SELC+ RoLR RankMatch CrossSplit DISC DMLP UNICON
Builds on: RoLR DivideMix FixMatch MixMatch DMLP
Closest papers by meaning
- Rethinking Label Refurbishment: Model Robustness under Label Noise (AAAI 2023 · similarity 0.89)
- Two Wrongs Don’t Make a Right: Combating Confirmation Bias in Learning with Label Noise (AAAI 2023 · similarity 0.88)
- Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise (NeurIPS 2018 · similarity 0.88)
- Noise-Robust Learning from Multiple Unsupervised Sources of Inferred Labels (AAAI 2022 · similarity 0.87)
- Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model (AAAI 2021 · similarity 0.87)
- Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.