Research map / Label-efficient and robust learning / Noisy labels
Label Noise: Ignorance Is Bliss
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames label noise as posterior-drift domain adaptation, introduces relative signal strength (RSS), proves near-matching minimax bounds for noise-ignorant ERM, and shows feature extraction plus a linear NI-ERM classifier reaches SOTA on CIFAR-10N/100N.
Problem
Learning classifiers under multi-class, instance-dependent label noise has little theory on achievable excess risk and fundamental limits.
Approach
Introduces relative signal strength, a pointwise ratio of noisy-to-clean posterior margins, to characterize the transferable signal under posterior drift; shows empirical risk minimization ignoring noise (NI-ERM) is nearly minimax optimal, and that a linear probe on self-supervised features trained with NI-ERM beats prior CIFAR-N methods.
Key results
- FE+NI-ERM with DINOv2 features reaches 98.69% on CIFAR-10N Aggre and 83.17% on CIFAR-100N Noisy, exceeding the leaderboard methods ProMix (73.79%) on CIFAR-100N.
- On CIFAR-10N Worst labels, FE+NI-ERM gets 95.71% versus ProMix's 96.34%, so it does not win every column.
- Simulations show NI-ERM's accuracy stays near Bayes until a noise threshold of 0.5 for binary and 0.9 for 10-class symmetric noise.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DINOv2 features with a noise-ignorant linear classifier instead of ProMix | CIFAR-10N · CIFAR-10N Aggre, Rand1, Rand2, Rand3, and Worst | testing accuracy: Aggre: 97.65 ± 0.19 -> 98.69 ± 0.00; Rand1: 97.39 ± 0.16 -> 98.80 ± 0.00; Rand2: 97.55 ± 0.12 -> 98.65 ± 0.00; Rand3: 97 | Mixed |
| Use DINOv2 features with a noise-ignorant linear classifier instead of ProMix | CIFAR-100N · CIFAR-100N Noisy | testing accuracy: 73.79 ± 0.28 -> 83.17 ± 0.00 | Helped |
Surprising findings
- Ignoring label noise via plain ERM on a linear probe is nearly minimax optimal under the RSS framework.
- A linear classifier on frozen self-supervised features outperforms specialized noisy-label methods on CIFAR-N.
Limitations
- Experimental evaluation is limited, as the authors acknowledge.
- Theory relies on Natarajan dimension and assumption that the noisy Bayes classifier lies in the function class.
- Lower bound is loose by a 1/K gap and a logarithmic factor in the upper bound.
Experimental setup
Datasets: CIFAR-10 CIFAR-10N CIFAR-100N MNIST
Benchmarks: CIFAR-10N CIFAR-100N
Metrics: testing accuracy
Baselines: ProMix ILL PLS DivideMix ResNet-50 end-to-end training
Builds on: Tsybakov margin condition Natarajan dimension DINOv2
Closest papers by meaning
- Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise (AAAI 2021 · similarity 0.91)
- Resurfacing the Instance-only Dependent Label Noise Model through Loss Correction (ICLR 2026 · similarity 0.90)
- Confidence Scores Make Instance-dependent Label-noise Learning Possible (ICML 2021 · similarity 0.90)
- Revisiting Learning with Noisy Labels: Active Forgetting and Noise Suppression (CVPR 2026 · similarity 0.89)
- Understanding Domain Generalization: A Noise Robustness Perspective (ICLR 2024 · similarity 0.89)
- Learning from Noisy Labels via Conditional Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.