Research map / Learning theory and algorithms / Kernels and regression theory
On Hypothesis Transfer Learning of Functional Linear Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RKHS-distance-based offset transfer learning for scalar-on-function linear and generalized linear models, with a calibrated multi-source TL-FLR and an aggregation-based ATL-FLR, and establishes matching minimax rates.
Problem
Transfer learning for functional linear regression (and functional GLMs) where truncation-based high-dimensional TL methods are incompatible with infinite-dimensional smooth functional data and lack theory.
Approach
Measure source-target task similarity by RKHS distance between coefficient functions and build offset transfer learning on it; use sparse aggregation over candidate source sets identified by truncated RKHS norms to avoid negative transfer, with matching minimax lower and upper bounds.
Key results
- In simulations, ATL-FLR outperforms OFLR, Detect-TL and ATL-FLR(EW) across all numbers of true transferable sources from 0 to 20.
- On HAR classification across 30 volunteers, ATL-FGLM achieves the lowest misclassification rate for most targets (e.g., Volunteer 1: 4.387% vs 5.548% Non-TL).
- Theory: TL-FLR attains excess risk O(n_S^{-2r/(2r+1)} + n_0^{-2r/(2r+1)} xi(h,R)) with a matching lower bound; ATL-FLR adds a log(T)log(n0)/n0 aggregation cost.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ATL-FLR with sparse aggregation instead of ATL-FLR(EW) | Functional linear regression transfer learning · Synthetic simulations | excess risk | Helped |
| Use ATL-FLR rather than ATL-FLR(EW), which is more sensitive to non-transferable sources | Functional linear regression transfer learning · Synthetic simulations with few positive sources | excess risk | Helped |
| Use ATL-FLR rather than direct transfer methods | Stock sector return prediction · Stock sector test sets | prediction error | Helped |
| Calibrate the pooled source estimator | TL-FLR on stock sector returns · Stock sector test sets | prediction error | Helped |
| Transfer from all sources without selecting positive sources | Pooled-TL and Naive-TL · Stock sector test sets | prediction error | Mixed |
| Use ATL-FGLM with sparse aggregation rather than competing transfer methods | Functional logistic regression · HAR volunteer test sets | misclassification rate | Helped |
| Transfer with known positive sources using TL-FLR | Functional linear regression · Synthetic simulations | relative excess risk versus OFLR | Mixed |
| Select sources using Detect-TL | Functional linear regression transfer learning · Synthetic simulations with large h | excess risk | No clear effect |
Surprising findings
- Pooled-TL and Naive-TL reduce error in only a few sectors and often worsen predictions, indicating random effects of direct transfer.
- The effectiveness of transfer depends on the signal strength of the source-target offset, not only on the RKHS distance h, so more sources can hurt unless carefully selected.
Limitations
- Optimal rates require oracle conditions: correctly specified hypothesis space and known smoothness for regularization.
- Data fusion transfer step has O(n_S^3) time and O(n_S^2) space, limiting scalability to large source samples.
- Identifiability assumption requires a finite truncation M so that non-transferable sources are separated; theory relies on commutativity or Hilbert-Schmidt conditions on operators.
Experimental setup
Datasets: Yahoo Finance stock prices (11 sectors) Human Activity Recognition (HAR) dataset synthetic Gaussian process data
Benchmarks: synthetic FLR simulations stock sector return prediction HAR misclassification
Metrics: excess risk relative excess risk relative prediction error misclassification rate
Baselines: OFLR Detect-TL ATL-FLR (EW) Pooled-TL Naive-TL Non-TL
Builds on: offset transfer learning kernel ridge regression sparse aggregation Matérn kernel
Closest papers by meaning
- Transfer Learning Meets Functional Linear Regression: No Negative Transfer Under Posterior Drift (AAAI 2025 · similarity 0.90)
- Wasserstein Transfer Learning (NeurIPS 2025 · similarity 0.87)
- Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression (NeurIPS 2025 · similarity 0.87)
- Smoothness Adaptive Hypothesis Transfer Learning (ICML 2024 · similarity 0.86)
- Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spaces (ICML 2025 · similarity 0.86)
- Transfer Learning of Linear Regression with Multiple Pretrained Models: Benefiting from More Pretrained Models via Overparameterization Debiasing (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.