Research map / Label-efficient and robust learning / Semi-supervised learning
Prediction-Powered Semi-Supervised Learning with Online Power Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends prediction-powered inference to SSL training by using a PPI-style unbiased gradient whose interpolation weight λ is tuned online with AdaGrad alongside the model weights.
Problem
Semi-supervised training with pseudo-labels from a teacher can be biased when the teacher is inaccurate on subgroups, and the optimal PPI interpolation weight depends on unknown quantities and is chosen offline, with no finite-time convergence guarantees for existing PPI-based SSL methods.
Approach
Builds an unbiased prediction-powered gradient estimator g^λ = g^n + λ(g̃^{N,f} − g^{n,f}) and treats λ as a trainable scalar updated online via 1D AdaGrad on the squared gradient norm, together with AdaGrad-norm model updates, so λ adapts to teacher quality without knowing variances.
Key results
- On synthetic data, PP-SSL achieves best group B MSE, especially at high teacher bias, and outperforms PPI++ and SSL.
- On California Housing, PPI++ and PP-SSL have comparable MSE and both outperform other baselines when the teacher is less accurate.
- On UTKFace, SSL performs worse than Only Labeled while PP-SSL achieves lower MAE with a noticeable advantage over PPI++.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PP-SSL with online tuning of λ instead of classic SSL pseudo-labeling | Semi-supervised training with a biased teacher · Synthetic linear regression, Group B | MSE | Helped |
| Use PP-SSL with online tuning of λ instead of PPI++ with offline tuning | Semi-supervised training with a biased teacher · Synthetic linear regression, Group B | MSE | Helped |
| Use SSL on labeled and pseudo-labeled data instead of training on labeled data only | Semi-supervised training with a biased teacher · Synthetic linear regression, Group B | MSE | Hurt |
| Use PP-SSL with online tuning of λ instead of a fixed λ | PPI-style prediction-powered gradient training · Synthetic linear regression, full test set | MSE | Helped |
| Use PP-SSL instead of classic SSL pseudo-labeling | Linear regression with a biased teacher · California Housing, test set | MSE | Helped |
| Use PP-SSL instead of PPI++ with offline tuning | Linear regression with a biased teacher · California Housing, test set | MSE | No clear effect |
| Use PP-SSL instead of labeled-only training | ResNet50 age estimation · UTKFace, test set | MSE | Helped |
| Use PP-SSL instead of classic SSL pseudo-labeling | ResNet50 age estimation · UTKFace, test set | MSE | Helped |
| Use PP-SSL instead of the evaluated baselines | Linear classifier on pretrained ResNet50 features · CIFAR-10 and CIFAR-10-C, Group B corrupted images | accuracy | Helped |
| Use PP-SSL instead of the evaluated baselines | Linear classifier on pretrained ResNet50 features · CIFAR-10 and CIFAR-10-C, Group A clean images | accuracy | No clear effect |
Surprising findings
- Standard SSL with pseudo-labels can perform worse than training only on labeled data when the teacher is biased on a subgroup.
Limitations
- Assumes labeled and unlabeled data come from the same input distribution.
- Theoretical analysis assumes a fixed teacher; self-training with evolving teachers is not covered.
Experimental setup
Datasets: synthetic linear regression data California Housing UTKFace CIFAR-10 CIFAR-10-C
Benchmarks: California Housing UTKFace CIFAR-10 CIFAR-10-C
Metrics: MSE MAE R^2 accuracy
Baselines: Teacher Only Labeled SSL (pseudo-label) PPI++ with optimal offline lambda
Builds on: Prediction-Powered Inference PPI++ AdaGrad Doubly-Robust Self-Training
Closest papers by meaning
- Regression for the Mean: Auto-Evaluation and Inference with Few Labels through Post-hoc Regression (ICML 2025 · similarity 0.91)
- Anytime-valid, Bayes-assisted, Prediction-Powered Inference (NeurIPS 2025 · similarity 0.89)
- Semi-Supervised Regression with Heteroscedastic Pseudo-Labels (NeurIPS 2025 · similarity 0.88)
- Bi-Level Optimization for Semi-Supervised Learning with Pseudo-Labeling (AAAI 2025 · similarity 0.87)
- Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.87)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.