Research map / Label-efficient and robust learning / Domain adaptation and generalization

A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning

NeurIPS 2025 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Statistical transfer learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes OTQMS, a K-L divergence based high-dimensional statistical framework that picks the optimal number of source samples per source task for joint training, and a dynamic resampling algorithm that re-selects source data each epoch.

Problem

Multi-source transfer learning typically uses all available source samples, which costs training efficiency and can cause negative transfer via bias from source-target discrepancy; the optimal transfer quantity per source is unknown.

Approach

Measure generalization error as the expected K-L divergence between target distribution and the MLE-learned distribution, derive its asymptotic form as a variance term plus a bias term depending on the total transfer quantity and a Fisher-information-weighted parameter distance, minimize it over total quantity and source proportions, and use pretrained source models as parameter proxies with dynam

Key results

• On DomainNet (10-shot, ViT-S), OTQMS averages 55.8% vs 54.3% for AllSources ∪ Target, a 1.5-point gain.
• On Office-Home, OTQMS averages 78.2% vs 77.2% for AllSources ∪ Target (1.0-point gain).
• OTQMS reduces average training time by 35.19% and average sample usage by 47.85% on DomainNet relative to AllSources ∪ Target.

Measured effects reported in the paper

Surprising findings

• Using all source samples can hurt: static transfer quantity over-fitting (Static-Over) yields 71.5% vs 77.2% on Office-Home.
• Unsupervised methods using all target data (MSFDA, M3SDA) underperform few-shot supervised methods with far fewer labeled samples.

Limitations

• Theory relies on asymptotic approximations and assumptions such as close parameters (|theta0 - theta1| = O(1/sqrt(N0))) and asymptotically comparable sample sizes.
• Uses pretrained source parameters as proxies for true source parameters and an empirical Fisher estimate, and reports no error bars.
• Evaluated only on image classification benchmarks with few-shot targets.

Experimental setup

Datasets: DomainNet Office-Home Digits

Benchmarks: DomainNet Office-Home

Metrics: accuracy training time sample usage

Baselines: MSFDA DATE M3SDA H-ensemble MADA MCW WADN Target-Only Single-Source-Avg Single-Source-Best AllSources ∪ Target

Builds on: Asymptotic normality of MLE Sanov's theorem empirical Fisher MADA H-ensemble

Closest papers by meaning

• A Mathematical Framework for Quantifying Transferability in Multi-source Transfer Learning (NeurIPS 2021 · similarity 0.90)
• H-ensemble: An Information Theoretic Approach to Reliable Few-Shot Multi-Source-Free Transfer (AAAI 2024 · similarity 0.88)
• Mixed-Sample SGD: an End-to-end Analysis of Supervised Transfer Learning (NeurIPS 2025 · similarity 0.88)
• Learning New Tricks From Old Dogs: Multi-Source Transfer Learning From Pre-Trained Networks (NeurIPS 2019 · similarity 0.88)
• Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learning (NeurIPS 2025 · similarity 0.86)
• Transfer Learning of Linear Regression with Multiple Pretrained Models: Benefiting from More Pretrained Models via Overparameterization Debiasing (NeurIPS 2026 · similarity 0.86)