Research map / Learning theory and algorithms / Kernels and regression theory
Smoothness Adaptive Hypothesis Transfer Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SATL, a two-phase kernel ridge regression offset transfer learning method using fixed-bandwidth Gaussian kernels with training/validation adaptation to unknown Sobolev smoothness, with matching minimax upper and lower bounds up to log factors.
Problem
Offset transfer learning in nonparametric regression uses the same kernel regularization in both phases and requires known smoothness, so it cannot adapt to unknown and differing smoothness of the target/source functions and their offset.
Approach
Shows that a fixed-bandwidth Gaussian kernel in target-only KRR attains minimax rates with lambda decaying exponentially in n, and uses Gaussian kernels in both phases of offset transfer with training/validation selection of smoothness, yielding adaptive minimax-optimal rates up to log factors.
Key results
- Target-only KRR with fixed-bandwidth Gaussian kernel and exponentially decaying lambda achieves rate n^{-2alpha0/(2alpha0+d)} (non-adaptive), with a training/validation variant adaptive up to a log factor.
- SATL's upper bound matches the minimax lower bound n_S^{-2m0/(2m0+d)} + n_T^{-2m/(2m+d)} xi(h,f_S) up to log factors.
- In synthetic experiments, SATL achieves smaller error than the FBE transfer baseline and outperforms target-only KRR in all settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SATL instead of target-only KRR | Target-only KRR · Synthetic transfer learning, varying n_T | generalization error | Helped |
| Use SATL instead of finite basis expansion with B-splines | Finite basis expansion with B-splines · Synthetic transfer learning, fixed n_T | generalization error | Helped |
| Increase source sample size n_S | SATL · Synthetic transfer learning, fixed n_T | generalization error | Helped |
| Use a smoother offset function | SATL · Synthetic transfer learning, fixed n_T | generalization error | Helped |
Surprising findings
- Optimal regularization for fixed-bandwidth Gaussian kernel KRR decays exponentially in n, unlike the polynomial pattern for Matérn misspecified kernels.
Limitations
- Results rely on Sobolev/norm-equivalent RKHS assumptions and do not cover structures such as periodic functions.
- Adaptivity incurs a logarithmic factor (price of adaptivity).
- Experiments are limited to synthetic one-dimensional data.
Experimental setup
Datasets: synthetic Gaussian process samples (Matérn covariance)
Benchmarks: synthetic 1D regression with Matérn GP-generated target and offset functions
Metrics: L2 generalization error fitted log-log convergence slope
Baselines: target-only KRR finite basis expansion TL (Fourier basis) finite basis expansion TL (B-spline) Matérn misspecified KRR
Builds on: offset transfer learning misspecified kernel ridge regression Lepski's method
Closest papers by meaning
- On Hypothesis Transfer Learning of Functional Linear Models (ICML 2024 · similarity 0.86)
- Kernel conditional tests from learning-theoretic bounds (NeurIPS 2025 · similarity 0.85)
- Kernel Regression in Structured Non-IID Settings: Theory and Implications for Denoising Score Learning (NeurIPS 2025 · similarity 0.85)
- Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression (NeurIPS 2025 · similarity 0.85)
- Transfer Learning Meets Functional Linear Regression: No Negative Transfer Under Posterior Drift (AAAI 2025 · similarity 0.85)
- Sparse Transfer Learning Accelerates and Enhances Certified Robustness: A Comprehensive Study (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.