Research map / Label-efficient and robust learning / Noisy labels
Dirichlet-based Per-Sample Weighting by Transition Matrix for Noisy Label Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RENT, a resampling-based way to use a noise transition matrix for noisy-label learning, unified with reweighting via a Dirichlet-based per-sample weight sampling (DWS) framework.
Problem
Learning with noisy labels using a transition matrix: prior work focuses on estimating the transition matrix, but how to utilize it (forward, backward, reweighting) has practical limitations, especially reweighting's dependence on an imperfect classifier's posterior estimates.
Approach
Introduces Dirichlet-based per-sample weight sampling (DWS), where reweighting (alpha to infinity) and resampling (alpha to 0) are special cases; analysis shows small alpha gives higher variance, closer distance to true weights, and noise-injection-like regularization. RENT resamples the noisy dataset with categorical probabilities proportional to the transition-corrected likelihood ratio.
Key results
- RENT outperforms forward loss in 42 of 48 CIFAR settings and outperforms reweighting in all but two cases.
- On Clothing1M with BLTM estimation, RENT reaches 70.0% versus 67.2% for FL and 67.0% for RW.
- Test accuracy increases as the Dirichlet shape parameter alpha decreases, with RENT best across tested alpha values.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Forward loss with RENT resampling | Cycle-estimated transition matrix · CIFAR10 SN 20% | test accuracy: 81.6 -> 82.5 | Helped |
| Replace Forward loss with RENT resampling | Cycle-estimated transition matrix · CIFAR10 SN 50% | test accuracy: 57.0$\pm$ -> 70.4 | Helped |
| Replace Forward loss with RENT resampling | Cycle-estimated transition matrix · CIFAR10 ASN 20% | test accuracy: 82.8 -> 81.5 | Hurt |
| Replace Forward loss with RENT resampling | Cycle-estimated transition matrix · CIFAR100 SN 20% | test accuracy: 39.9 -> 40.7 | Helped |
| Replace Forward loss with RENT resampling | BLTM-estimated transition matrix · CIFAR-10N Ran1 | test accuracy: 78.1 -> 79.1 | Helped |
| Replace Forward loss with RENT resampling | BLTM-estimated transition matrix · Clothing1M | test accuracy: 67.2 -> 70.0 | Helped |
| Use RENT instead of Reweighting | Transition-matrix utilization · CIFAR10 and CIFAR100 synthetic-noise settings | test accuracy | Mixed |
| Decrease the DWS concentration parameter α toward 0 (ablation) | DWS per-sample weighting · CIFAR10 SN 20% and SN 50% | test accuracy | Helped |
| Reduce the RENT resampling budget M (ablation) | RENT · CIFAR10 SN 20% and SN 50% | test accuracy | No clear effect |
| Use mini-batch resampling instead of full-dataset resampling (ablation) | RENT · Noisy-label classification | test accuracy | No clear effect |
| Add RENT resampling rather than using Forward loss alone | Transition-matrix utilization · CIFAR10 and CIFAR100 synthetic-noise settings | test accuracy | Helped |
| Use RENT instead of label perturbation alone (SNL) | Noisy-label learning · CIFAR10 SN 20% and SN 50% | test accuracy | Helped |
Surprising findings
- Reweighting with the Forward-estimated T shows near-failure cases (e.g., BLTM on CIFAR-10N gives 54.0% under aggregate noise for RW), indicating instability.
- Under true T, reweighting sometimes fails to train (reported as training failure on ASN settings).
Limitations
- Theoretical consistency of RENT relies on the true mean weights mu* being accessible; in practice T must be estimated.
- Resampling budget M and mini-batch resampling introduce practical choices; RENT underperforms RW in a couple of cases with small gaps.
- Analysis of alpha effects is partly heuristic (variance and Mahalanobis distance arguments).
Experimental setup
Datasets: CIFAR10 CIFAR100 CIFAR-10N Clothing1M
Benchmarks: CIFAR10 CIFAR100 CIFAR-10N Clothing1M
Metrics: test accuracy precision recall F1 score
Baselines: Forward (FL) Reweighting (RW) DualT TV VolMinNet Cycle PDN BLTM FINE MCD SNL
Builds on: Forward loss Backward loss reweighting (importance sampling) Sampling-Importance-Resampling SNL
Closest papers by meaning
- Estimation of the Label-Noise Transition Matrix with Performance Guarantees via Selective Classification (NeurIPS 2026 · similarity 0.87)
- Learning the Latent Causal Structure for Modeling Label Noise (NeurIPS 2024 · similarity 0.86)
- Learning with Noisy Labels Using Hyperspherical Margin Weighting (AAAI 2024 · similarity 0.85)
- Label-Noise Robust Diffusion Models (ICLR 2024 · similarity 0.85)
- Learning Causal Transition Matrix for Instance-dependent Label Noise (AAAI 2025 · similarity 0.85)
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.