Research map / Label-efficient and robust learning / Semi-supervised learning
SemiReward: A General Reward Model for Semi-supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SemiReward, a pluggable reward model that predicts cosine-similarity-based reward scores to filter pseudo labels in semi-supervised learning, trained online in two stages with a generator, and reports gains and faster convergence on classification and regression SSL benchmarks.
Problem
Selecting high-quality pseudo labels in semi-supervised learning is hampered by confidence-based hand-crafted policies that are task-specific (classification only) and by confirmation bias, limiting quality, convergence speed, and task versatility.
Approach
Model pseudo-label quality as a reward score (scaled cosine similarity between pseudo and ground-truth labels) predicted by a lightweight rewarder that cross-attends over unlabeled inputs and pseudo labels; train the rewarder online in two stages with a generator producing decoupled "fake labels" and random subsampling to mitigate confirmation bias.
Key results
- Plugging SemiReward into Pseudo Label yields +4.11% average gain on ESC-50 (250 labels) and convergence speedups of about 1.3x to 3.53x in most settings.
- On RCF-MNIST regression, SemiReward lowers RMSE to 0.9 and MAE to 0.99, better than the supervised baseline.
- SemiReward adds only 1.28% parameters and 0.056% FLOPs relative to the ViT-S student on CIFAR-100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SemiReward to the strongest listed baseline | SSL classification methods · ESC-50 (250 labels) | top-1 error rate (%): 32.71{\scriptsize $\pm$0.82} -> 29.71{\scriptsize $\pm$0.64} | Helped |
| Add SemiReward to FlexMatch | FlexMatch · ESC-50 (500 labels) | top-1 error rate (%): 27.75{\scriptsize $\pm$0.41} -> 25.92{\scriptsize $\pm$0.31} | Helped |
| Add SemiReward to FlexMatch | FlexMatch · FSDnoisy18k (1773 labels) | top-1 error rate (%): 26.29{\scriptsize $\pm$0.17} -> 25.63{\scriptsize $\pm$0.28} | Helped |
| Add SemiReward to SoftMatch/FreeMatch | SoftMatch/FreeMatch · AG News (40 labels) | top-1 error rate (%): 11.69{\scriptsize $\pm$0.12} -> 10.67{\scriptsize $\pm$0.90} | Helped |
| Add SemiReward to SoftMatch/FreeMatch | SoftMatch/FreeMatch · Yahoo! Answer (500 labels) | top-1 error rate (%): 33.02{\scriptsize $\pm$0.02} -> 30.92{\scriptsize $\pm$0.90} | Helped |
| Add SemiReward to FlexMatch | FlexMatch · Yelp Review (1000 labels) | top-1 error rate (%): 40.38{\scriptsize $\pm$0.33} -> 37.58{\scriptsize $\pm$0.19} | Helped |
| Add SemiReward to FlexMatch | FlexMatch · STL-10 (100 labels) | top-1 error rate (%): 7.13{\scriptsize $\pm$0.20} -> 7.59{\scriptsize $\pm$0.57} | No clear effect |
| Start stage 2 at 10% rather than 0% (ablation) | SemiReward · CIFAR-100 (400 labels) | accuracy (%): 19.65 -> 16.65 | Helped |
| Replace MSE with BCE reward loss (ablation) | SemiReward · CIFAR-100 (400 labels) | accuracy (%): 16.65 -> 17.66 | Hurt |
| Remove cross-attention from the rewarder (ablation) | SemiReward rewarder · CIFAR-100 (400 labels) | accuracy (%): 83.26 -> 82.20 | Hurt |
| Increase rewarder MLP depth from 1 to 2 layers (ablation) | SemiReward rewarder · CIFAR-100 (400 labels) | accuracy (%): 83.35 -> 83.26 | No clear effect |
| Use negative L2 distance instead of cosine similarity as reward target (ablation) | SemiReward · CIFAR-100 (400 labels) | accuracy (%): 83.35 -> 80.23 | Hurt |
Surprising findings
- Coupled training of the rewarder with the student (without the generator) yields worse performance than baseline (82.12 vs 82.20 on CIFAR-100 with FlexMatch+SR).
- CRMatch performs poorly across regression datasets, inferior to other SSL baselines, which the authors attribute to confirmation bias.
Limitations
- Reward scores and rewarder only support sample-level labels, not fine-grained labels such as detection.
- Requires repeated teacher and rewarder forward passes to obtain reliable pseudo labels, adding extra computation.
- Underperforms on imbalanced regression datasets (IMDB-WIKI, AgeDB), where it struggles to improve pseudo-label selection.
Experimental setup
Datasets: CIFAR-100 STL-10 EuroSAT ImageNet AG News Yahoo! Answers Yelp Review UrbanSound8k ESC-50 FSDNoisy18k IMDB-WIKI AgeDB
Benchmarks: CIFAR-100 STL-10 EuroSAT ImageNet AG News Yahoo! Answers Yelp Review UrbanSound8k ESC-50 FSDNoisy18k IMDB-WIKI AgeDB
Metrics: top-1 error rate RMSE MAE training speedup
Baselines: Pseudo Label FlexMatch SoftMatch FreeMatch CRMatch Mean Teacher Pi model
Builds on: FlexMatch SoftMatch FreeMatch USB Pseudo Label CRMatch
Closest papers by meaning
- Bi-Level Optimization for Semi-Supervised Learning with Pseudo-Labeling (AAAI 2025 · similarity 0.89)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.88)
- Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL (ICML 2025 · similarity 0.87)
- Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.87)
- Reinforcement Learning Guided Semi-Supervised Learning (NeurIPS 2024 · similarity 0.87)
- Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.