Research map / Large language models / Language, safety and interpretability
Self Iterative Label Refinement via Robust Unlabeled Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an iterative pipeline that refines LLM-generated pseudo-labels for classification using robust Unlabeled-Unlabeled (UU) learning on two pseudo-labeled sets with different positive priors, and shows gains over GPT-4o/DeepSeek-R1 self-refinement and an RLAIF safety reward model.
Problem
LLM self-refinement and self-evaluation are unreliable in domains where the model lacks internal knowledge, and LLM-generated pseudo-labels are noisy, so classification and RLAIF degrade without costly human labels.
Approach
Treat LLM pseudo-positive and pseudo-negative subsets as two unlabeled sets with different positive priors and train a classifier with robust UU learning (generalized leaky ReLU on negative risk terms), then re-label the whole corpus iteratively; prior estimates come from oracle values or 50-100 labeled examples.
Key results
- On Safety with Gemma-2-2b initial labels, Ours (Oracle) reaches about 0.842 accuracy at iteration 5 vs 0.697 at iteration 0 and 0.777 for UU.
- On Protein Structure with GPT-4o labels, Ours (Oracle) climbs from 0.545 to about 0.777 over iterations, while GPT-4o self-refinement falls to 0.599.
- Ours with only 50 labeled examples for prior estimation reaches accuracy comparable to the oracle-prior variant on easier tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add robust correction to standard UU learning (ablation) | Iterative label refinement · Saroco | accuracy: 0.903 vs 0.943 | Helped |
| add robust correction to standard UU learning (ablation) | Iterative label refinement · Safety | accuracy: 0.795 vs 0.826 | Helped |
| estimate class priors from 50 labeled examples instead of using exact priors (ablation) | Iterative robust UU learning · Saroco | accuracy: 0.943 vs 0.942 | No clear effect |
| replace DeepSeek-R1 self-refinement with iterative robust UU refinement | Classification on Corona Sentiment · Corona Sentiment | accuracy: 0.773 -> 0.846 | Helped |
| replace DeepSeek-R1 self-refinement with iterative robust UU refinement | Classification on Green Patent · Green Patent | accuracy: 0.662 -> 0.813 | Helped |
| replace DeepSeek-R1 self-refinement with iterative robust UU refinement | Classification on Protein Structure · Protein Structure | accuracy: 0.707 -> 0.777 | Helped |
| iteratively self-refine with GPT-4o | GPT-4o classification on Green Patent · Green Patent | accuracy: 0.676 -> 0.611 | Hurt |
| iteratively self-refine with GPT-4o | GPT-4o classification on Protein Structure · Protein Structure | accuracy: 0.545 -> 0.599 | Helped |
| iteratively self-refine with GPT-4o | GPT-4o classification on Corona Sentiment · Corona Sentiment | accuracy: 0.710 -> 0.806; 0.710 -> 0.772 | Mixed |
Surprising findings
- Self-refinement with GPT-4o and DeepSeek-R1 fails to improve or degrades accuracy on the harder tasks.
- Vanilla RLAIF with a PN-trained reward model performs worse than the SFT baseline on the Safety alignment task.
Limitations
- Performance degrades when initial pseudo-label noise is extreme (near-identical positive priors across the two sets).
- Prior estimated from 50 examples may be biased if those examples are out-of-distribution.
- Only applied to binary classification pseudo-labels; the LLM post-training application is not explored in depth.
Experimental setup
Datasets: Fake News Saroco Safety (SafeRLHF) Corona Sentiment Green Patent Protein Structure (binding_affinity) Alpaca
Benchmarks: Fake News Saroco Safety Corona Sentiment Green Patent Protein Structure
Metrics: classification accuracy reward distribution
Baselines: PN PIE CCP UU GPT-4o self-refinement GPT-4o-mini self-refinement DeepSeek-R1 self-refinement Vanilla RLAIF SFT
Builds on: unlabeled-unlabeled learning robust UU learning PIE CCP RLAIF
Closest papers by meaning
- Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning (AAAI 2024 · similarity 0.87)
- Learning from Noisy Labels via Self-Taught On-the-Fly Meta Loss Rescaling (AAAI 2025 · similarity 0.87)
- ULAREF: A Unified Label Refinement Framework for Learning with Inaccurate Supervision (ICML 2024 · similarity 0.86)
- OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning (NeurIPS 2024 · similarity 0.86)
- Distilling Reliable Knowledge for Instance-Dependent Partial Label Learning (AAAI 2024 · similarity 0.86)
- Iterative Label Refinement Matters More than Preference Optimization under Weak Supervision (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.