Research map / Label-efficient and robust learning / Noisy labels
Handling Label Noise via Instance-Level Difficulty Modeling and Dynamic Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IDO, a two-stage noisy-label learning framework that uses a cheap "wrong event" counter and a two-component beta mixture model to derive per-sample cleanliness and difficulty weights for a dynamically weighted loss, without extra hyperparameters.
Problem
Deep networks overfit label noise, and existing noisy-label learning methods are computationally expensive, hyperparameter-heavy, and apply coarse category-level weighting that ignores per-sample differences in cleanliness and difficulty.
Approach
Replace loss-based metrics with "wrong event" (cumulative count of epochs where prediction disagrees with given label), fit per-class two-component beta mixture models to it, and use posterior and CDF values as instance-level weights for clean, noisy, and difficult-sample loss terms, updated every epoch.
Key results
- On CIFAR-100 with ViT-B/16 (Sym. 60%), IDO reaches 91.45% vs. 89.74% for the best listed baseline (ELR) and 89.42% for DeFT.
- On real-world CIFAR-100N with ResNet-50, IDO achieves 73.58% vs. 73.33% for DISC and 72.91% for DMix.
- On ViT-B/16 CIFAR-100 Inst. 40%, IDO takes 458 s per epoch and 14.8 GB vs. DMix 1596 s / 31.8 GB and UNICON 2628 s / 45.2 GB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the noisy-sample loss L_N to L_C (ablation) | IDO · CIFAR-100 Sym. 60% | accuracy: 75.1 -> 79.5 | Helped |
| Add the noisy-sample loss L_N to L_C (ablation) | IDO · CIFAR-100 Asym. 40% | accuracy: 57.7 -> 70.4 | Helped |
| Add the noisy-sample loss L_N to L_C (ablation) | IDO · CIFAR-100 Inst. 40% | accuracy: 70.5 -> 77.3 | Helped |
| Add the difficulty consistency loss L_SIM to L_C (ablation) | IDO · CIFAR-100 Sym. 60% | accuracy: 75.1 -> 78.2 | Helped |
| Add the difficulty consistency loss L_SIM to L_C (ablation) | IDO · CIFAR-100 Asym. 40% | accuracy: 57.7 -> 72.0 | Helped |
| Add the difficulty consistency loss L_SIM to L_C (ablation) | IDO · CIFAR-100 Inst. 40% | accuracy: 70.5 -> 80.3 | Helped |
| Use dynamic difficulty weighting instead of no difficulty weight (ablation) | IDO · CIFAR-100 Sym. 60% | accuracy: 79.5 -> 81.4 | Helped |
| Use dynamic difficulty weighting instead of no difficulty weight (ablation) | IDO · CIFAR-100 Asym. 40% | accuracy: 70.4 -> 78.2 | Helped |
| Use dynamic difficulty weighting instead of no difficulty weight (ablation) | IDO · CIFAR-100 Inst. 40% | accuracy: 77.3 -> 83.8 | Helped |
| Use dynamic difficulty weighting instead of fixed ε=0.25 (ablation) | IDO · CIFAR-100 (Sym. 60%, Asym. 40%, Inst. 40%) | accuracy: 81.3 -> 81.4; 76.5 -> 78.2; 83.3 -> 83.8 | Helped |
| Use dynamic difficulty weighting instead of fixed ε=0.5 (ablation) | IDO · CIFAR-100 (Sym. 60%, Asym. 40%, Inst. 40%) | accuracy: 80.8 -> 81.4; 77.5 -> 78.2; 82.4 -> 83.8 | Helped |
| Use dynamic difficulty weighting instead of fixed ε=1 (ablation) | IDO · CIFAR-100 (Sym. 60%, Asym. 40%, Inst. 40%) | accuracy: 80.3 -> 81.4; 76.8 -> 78.2; 82.8 -> 83.8 | Helped |
Surprising findings
- Loss-based selection performs well early in training but degrades sharply later due to noise memorization, while wrong event stays discriminative across all stages.
- Hyperparameter sensitivity analysis shows ELR and DivideMix accuracy varies widely with their hyperparameters (e.g., ELR drops to 1.0% at lambda=10 on CIFAR-100 Sym. 60%).
Limitations
- Performance becomes suboptimal when noise ratio exceeds about 90%, as the wrong-event distribution becomes extremely imbalanced.
- Relies on CLIP-free pretrained backbones; DeFT-style baselines are noted as resource-heavy, but the method's sensitivity to stage-1 duration and pretraining choices is not fully characterized.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet CIFAR-100N Clothing1M WebVision
Benchmarks: CIFAR-100 Tiny-ImageNet CIFAR-100N Clothing1M WebVision
Metrics: test accuracy average per-epoch time (s) peak GPU memory (GB) F-score AUC
Baselines: SCE DivideMix ELR Co-learning UNICON DISC TURN DeFT LongReMix ProMix
Builds on: DivideMix ELR forgetting event label wave DeFT
Closest papers by meaning
- Learning with Instance-Dependent Label Noise: A Sample Sieve Approach (ICLR 2021 · similarity 0.90)
- Robust Classification via Regression for Learning with Noisy Labels (ICLR 2024 · similarity 0.90)
- Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model (AAAI 2021 · similarity 0.90)
- Mitigating Label Noise through Data Ambiguation (AAAI 2024 · similarity 0.90)
- NoiseGPT: Label Noise Detection and Rectification through Probability Curvature (NeurIPS 2024 · similarity 0.89)
- On the Role of Label Noise in the Feature Learning Process (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.