Research map / Label-efficient and robust learning / Semi-supervised learning

Semi-Supervised Regression with Heteroscedastic Pseudo-Labels

NeurIPS 2025 · Label-efficient and robust learning › Semi-supervised learning · approach: Confidence-based pseudo-label selection · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an uncertainty-aware pseudo-labeling framework for semi-supervised regression that learns per-sample pseudo-label log-variances via a bi-level optimization, with an efficient regression-head-only approximation.

Problem

Pseudo-labeling for semi-supervised regression is hard because continuous pseudo-labels carry heteroscedastic noise that is difficult to assess, so naive pseudo-labels cause error accumulation and overfitting to incorrect labels.

Approach

An uncertainty-learner network outputs per-pseudo-label log-variance and is optimized in an outer loop to minimize labeled-data loss of the regression model updated by an inner heteroscedastic weighted loss over labeled and pseudo-labeled data, with gradient unrolling restricted to the regression head.

Key results

• On UTKFace with 5% labels, MAE 5.639 vs. 5.719 for second-best RankUp (about 1.4% relative improvement).
• On IMDB-WIKI with 5% labels, MAE 9.177 and R^2 0.664, versus 9.730 MAE for UCVME (best of prior baselines near that setting).
• On STS-B with 5% labels, MSE 1.540 vs. 1.606 for SSDKL, with R^2 0.270 vs. 0.239; method adds about 9 ms per iteration and about 17 MB GPU memory over baseline on UTKFace.

Measured effects reported in the paper

Surprising findings

• The uncertainty-learner used jointly (without bi-level optimization) performs worse than the baseline with fixed uncertainty on IMDB-WIKI.
• On UTKFace at 20% labels, UCVME outperforms the proposed method, which the authors attribute to stronger supervision improving pseudo-label quality.

Limitations

• Does not explicitly address systematic biases such as demographic imbalance in labeled or unlabeled data.
• Assumes labeled and unlabeled data are accessible simultaneously during training, raising privacy concerns.
• Gains diminish as the labeled fraction grows; UCVME and RankUp are better on UTKFace at 20% labels.

Experimental setup

Datasets: UTKFace IMDB-WIKI STS-B

Benchmarks: UTKFace IMDB-WIKI STS-B

Metrics: MAE R^2 MSE

Baselines: Supervised Mean Teacher Temporal Ensembling SSDKL TNNR SimRegMatch UCVME CLSS RankUp PIM UPS_Reg CADR

Builds on: FixMatch UCVME Mean Teacher Temporal Ensembling

Closest papers by meaning

• Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.91)
• Bi-Level Optimization for Semi-Supervised Learning with Pseudo-Labeling (AAAI 2025 · similarity 0.90)
• Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL (ICML 2025 · similarity 0.88)
• Prediction-Powered Semi-Supervised Learning with Online Power Tuning (NeurIPS 2025 · similarity 0.88)
• Learning from Noisy Labels via Conditional Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.87)
• Non-Stationary Predictions May Be More Informative: Exploring Pseudo-Labels with a Two-Phase Pattern of Training Dynamics (ICML 2025 · similarity 0.87)