Research map / Learning theory and algorithms / Prediction and decision losses

Transfer Learning Meets Functional Linear Regression: No Negative Transfer Under Posterior Drift

AAAI 2025 · Learning theory and algorithms › Prediction and decision losses · approach: Distributionally robust optimization · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a transfer learning estimator for functional linear regression slope under posterior drift (shared eigenspace) using FPCA scores, approximated least squares, and a lasso bias correction, plus a sparse-aggregation adaptive variant that guards against non-informative sources.

Problem

Slope estimation in functional linear regression when target and source data have different response-covariate relationships (posterior drift) and possibly different covariate distributions, with the risk of negative transfer.

Approach

Project target and source functional covariates onto a common eigenspace estimated from pooled source data, get an initial slope from source scores via approximated least squares, then correct bias on target data with a lasso penalty; an adaptive version selects candidate source sets via cross-covariance discrepancy and combines estimators by sparse aggregation.

Key results

• In aligned-eigenspace simulations (Model I), A_h TL-FLR outperforms FLR across all tested settings, including the large contrast h=2000 in the supplement.
• In Model IV (unaligned eigenspaces), Agg TL-FLR remains comparable to or slightly worse than FLR while Naive TL-FLR and A_h TL-FLR are worse than FLR.
• On real stock data, Agg TL-FLR has relative prediction errors significantly below 1 in almost all sectors while Naive TL-FLR often performs much worse than FLR.

Measured effects reported in the paper

Surprising findings

• Naive TL-FLR and A_h TL-FLR perform worse than FLR when all source eigenspaces are misaligned (Model IV).
• Sparse aggregation outperforms Q-aggregation under Models III and IV, while the two are comparable under Model II.

Limitations

• Theory requires shared eigenfunctions (posterior drift with aligned covariance eigenspaces) for the main estimator; performance degrades when eigenspaces are unaligned.
• Requires sufficiently large source sample size and small contrast for improvement over target-only estimation.
• Tuning parameters (truncation m and lasso tau) selected by cross-validation without theory for data-driven choice.

Experimental setup

Datasets: simulated functional data (Models I-IV) stock price data 05/01/2021-09/30/2021 across 11 sectors

Benchmarks: simulated functional linear regression models I-IV stock price prediction across 11 sectors

Metrics: mean integrated squared error relative prediction error

Baselines: FLR (FPCA-based estimation using target data only) Naive TL-FLR Q-aggregation variant exponential aggregation

Builds on: functional linear regression FPCA estimator lasso sparse aggregation exponential aggregation Q-aggregation

Closest papers by meaning

• On Hypothesis Transfer Learning of Functional Linear Models (ICML 2024 · similarity 0.90)
• Transfer Learning of Linear Regression with Multiple Pretrained Models: Benefiting from More Pretrained Models via Overparameterization Debiasing (NeurIPS 2026 · similarity 0.88)
• Wasserstein Transfer Learning (NeurIPS 2025 · similarity 0.86)
• Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression (NeurIPS 2025 · similarity 0.86)
• Smoothness Adaptive Hypothesis Transfer Learning (ICML 2024 · similarity 0.85)
• Minimum-Norm Interpolation Under Covariate Shift (ICML 2024 · similarity 0.84)