Research map / Label-efficient and robust learning / Domain adaptation and generalization

Pseudo-Labeling for Unsupervised Domain Adaptation with Kernel GLMs

NeurIPS 2026 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Conditional distribution alignment · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes pseudo-labeling model selection for unsupervised covariate-shift adaptation in ridge-regularized kernel GLMs, using a held-out source split to train an imputer and derives excess-risk bounds via an effective sample size.

Problem

Selecting the ridge regularization for kernel GLMs (linear, logistic, Poisson) under covariate shift when only unlabeled target covariates are available.

Approach

Split source data: one half trains a grid of candidate kernel GLMs, the other half trains an undersmoothed imputation model whose conditional-mean predictions serve as pseudo-labels on target covariates; the candidate minimizing pseudo-target GLM risk is selected. Non-asymptotic excess-risk bounds are expressed via an effective labeled sample size n_eff capturing covariate shift.

Key results

• On the Raisin shifted split, pseudo-labeling risk 0.428 (95% CI [0.412, 0.444]) vs naive 0.502 and oracle 0.373.
• Synthetic experiments: fitted decay exponents 0.546 (pseudo-labeling) vs 0.523 (oracle) vs 0.439 (naive) for n from 4000 to 32000.
• Theory: excess risk bound of order n_eff^{-2α/(2α+1)} + n_0^{-1} with overhead of pseudo-labels negligible.

Surprising findings

• Hard pseudo-labels are argued to introduce bias; soft labels are used instead, and the imputer is deliberately undersmoothed (lambda ~ 1/n).

Limitations

• Theory assumes fixed or specific kernel conditions (Mercer expansion, bounded eigenfunctions) and a well-specified model.
• Real-data experiment uses a single shift strength and a small dataset (900 instances).
• Pointwise rate O(n^{-1/4}) is slower than L2 rate, and the imputer tuning guidance is theory-driven rather than empirically validated across settings.

Experimental setup

Datasets: Raisin dataset (UCI) synthetic Sobolev-kernel logistic data

Benchmarks: Raisin dataset (covariate-shifted split) synthetic 1D logistic regression

Metrics: excess risk (log-loss) estimated convergence exponent

Baselines: Naive source hold-out validation Oracle selection with target labels

Builds on: kernel ridge regression pseudo-label regularization selection importance weighting Mercer eigendecay analysis

Closest papers by meaning

• Guiding Pseudo-Labels With Uncertainty Estimation for Source-Free Unsupervised Domain Adaptation (CVPR 2023 · similarity 0.90)
• Discretized Density-Guided Source-Free Domain Adaptation for Regression (ICML 2026 · similarity 0.90)
• Pseudo-Calibration: Improving Predictive Uncertainty Estimation in Unsupervised Domain Adaptation (ICML 2024 · similarity 0.88)
• A General Representation-Based Approach to Multi-Source Domain Adaptation (ICML 2025 · similarity 0.88)
• Revisiting Source-Free Domain Adaptation: Insights into Representativeness, Generalization, and Variety (CVPR 2025 · similarity 0.88)
• Can We Evaluate Domain Adaptation Models Without Target-Domain Labels? (ICLR 2024 · similarity 0.88)