Research map / Reinforcement learning / Policy gradient and actor-critic
Learning to Clean: Reinforcement Learning for Noisy Label Correction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates noisy label correction as an MDP and trains an actor-critic policy network that iteratively corrects training labels using kNN-based consistency and noisy-to-clean alignment rewards.
Problem
Learning with noisy labels, especially instance-dependent label noise, degrades deep network generalization; existing methods lack active exploration or sensitivity to long-term consequences of label corrections.
Approach
Treat label correction as a sequential decision process: a stochastic policy over kNN-attention label predictions decides which labels to correct, trained with actor-critic using rewards based on KL-divergence label consistency and alignment of corrected labels with clean neighbors.
Key results
- On CIFAR100-IDN at 0.50 noise, RLNLC reaches 74.7% test accuracy vs 72.8% for SSR.
- On Animal-10N, RLNLC achieves 90.2% vs 89.0% for SURE; on Food-101N 89.2% vs 87.3% for LongReMix.
- On CIFAR100 symmetric 90% noise, RLNLC reaches 44.2% vs 31.0% for DivideMix.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RLNLC instead of SSR | CIFAR10-IDN · CIFAR10-IDN, noise 0.50 | test accuracy (%): 95.8 vs 94.1 | Helped |
| Use RLNLC instead of SSR | CIFAR100-IDN · CIFAR100-IDN, noise 0.50 | test accuracy (%): 74.7 vs 72.8 | Helped |
| Use RLNLC instead of SURE | Animal-10N · Animal-10N | test accuracy (%): 90.2 vs 89.0 | Helped |
| Use RLNLC instead of LongReMix | Food-101N · Food-101N | test accuracy (%): 89.2 vs 87.3 | Helped |
| Use RLNLC instead of DivideMix | CIFAR100 with symmetric noise · CIFAR100, symmetric noise 0.90 | test accuracy (%): 44.2 vs 31.0 | Helped |
| Add the noisy label alignment reward (ablation) | RLNLC · CIFAR100-IDN, noise 0.50 | test accuracy (%): 72.0 -> 74.7 | Helped |
| Add the label consistency reward (ablation) | RLNLC · CIFAR100-IDN, noise 0.50 | test accuracy (%): 73.9 -> 74.7 | Helped |
| Add initial state randomization (ablation) | RLNLC · CIFAR100-IDN, noise 0.50 | test accuracy (%): 73.1 -> 74.7 | Helped |
| Use a separate f_omega reward feature extractor instead of f_theta (ablation) | RLNLC · CIFAR100-IDN, noise 0.50 | test accuracy (%): 73.8 -> 74.7 | Helped |
| Increase k from 3 to 10 (ablation) | RLNLC · CIFAR100-IDN, noise 0.50 | test accuracy (%) | Helped |
| Increase k beyond 10 to 20 or 30 (ablation) | RLNLC · CIFAR100-IDN, noise 0.50 | test accuracy (%): 74.9% | No clear effect |
| Increase N_b from 10 to 100 (ablation) | RLNLC · CIFAR100-IDN, noise 0.50 | test accuracy (%) | Helped |
Surprising findings
- Removing the random initial-state perturbation causes only a modest drop, while removing the policy/reward feature decoupling (f_omega <- f_theta) hurts more on CIFAR100-IDN at 0.50 noise (73.8 vs 74.7).
Limitations
- Relies on a pre-trained feature extractor and kNN label aggregation, so reward quality depends on initial embeddings.
- Performance drops with very long trajectories (T=25), suggesting overfitting or label drift.
- Results on CIFAR-IDN are reported with a combined pipeline (policy training then fine-tuning), making gains hard to attribute to RL alone.
Experimental setup
Datasets: CIFAR10-IDN CIFAR100-IDN Animal-10N Food-101N CIFAR10 CIFAR100
Benchmarks: CIFAR10-IDN CIFAR100-IDN Animal-10N Food-101N CIFAR10 CIFAR100
Metrics: test accuracy (%) label correction accuracy
Baselines: CE Mixup Forward Reweight Decoupling Co-teaching MentorNet DivideMix CausalNL SSR SURE PLC
Builds on: DivideMix SSR actor-critic SARSA
Closest papers by meaning
- Meta Label Correction for Noisy Label Learning (AAAI 2021 · similarity 0.86)
- Mitigating Label Noise through Data Ambiguation (AAAI 2024 · similarity 0.86)
- Error-Bounded Correction of Noisy Labels (ICML 2020 · similarity 0.86)
- Reinforcement Learning Guided Semi-Supervised Learning (NeurIPS 2024 · similarity 0.85)
- Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning (AAAI 2024 · similarity 0.85)
- Reinforcement Learning from Imperfect Corrective Actions and Proxy Rewards (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.