Research map / Label-efficient and robust learning / Domain adaptation and generalization
H-ensemble: An Information Theoretic Approach to Reliable Few-Shot Multi-Source-Free Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes H-ensemble, which learns linear source-feature weights by maximizing a one-sided H-score (generalized maximal correlation regression) for few-shot multi-source-free transfer using only black-box source feature extractors.
Problem
Few-shot transfer to a target task from multiple source models when neither source data nor source model details (intermediate layers, weights) are accessible.
Approach
Model the target feature extractor as a convex combination of source feature extractors, set the optimal target classifier in closed form as the weighted class-conditional feature means, and optimize the weights by maximizing a one-sided H-score, which is a convex quadratic in the weights and solvable with projected gradient descent.
Key results
- On VisDA-2017 10-shot, H-ensemble averages 0.8986 accuracy over the 8 target tasks, versus 0.8956 for the best single-source baseline and 0.8805 for MCW.
- On 8-shot Office datasets, H-ensemble reaches 0.8644 on Office-31 and 0.6940 on Office-Home, outperforming Average Weights (0.8479 and 0.6702 respectively).
- DECISION and DATE perform poorly (avg 0.4478 and 0.3748 on VisDA-2017) when transferred from unsupervised DA to few-shot settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use H-ensemble instead of Single-Best | few-shot multi-source-free transfer · VisDA-2017 10-shot (Avg.) | accuracy: 0.8956 -> 0.8986 | Helped |
| Add multi-source transfer instead of using average single-source performance (ablation) | H-ensemble · Office-31 8-shot | accuracy: 0.8483 -> 0.8644 | Helped |
| Add multi-source transfer instead of using average single-source performance (ablation) | H-ensemble · Office-Caltech 8-shot | accuracy: 0.9645 -> 0.9708 | Helped |
| Add multi-source transfer instead of using average single-source performance (ablation) | H-ensemble · Office-Home 8-shot | accuracy: 0.6235 -> 0.6940 | Helped |
| Use learned source weights instead of equal weights (ablation) | H-ensemble · Office-31 8-shot | accuracy: 0.8479 -> 0.8644 | Helped |
| Use learned source weights instead of equal weights (ablation) | H-ensemble · Office-Caltech 8-shot | accuracy: 0.9716 -> 0.9708 | Hurt |
| Use learned source weights instead of equal weights (ablation) | H-ensemble · Office-Home 8-shot | accuracy: 0.6702 -> 0.6940 | Helped |
| Use the MCR classifier instead of a finetuned classifier (ablation) | H-ensemble · Office-31 8-shot | accuracy: 0.8414 -> 0.8644 | Helped |
| Use the MCR classifier instead of a finetuned classifier (ablation) | H-ensemble · Office-Caltech 8-shot | accuracy: 0.9663 -> 0.9708 | Helped |
| Use the MCR classifier instead of a finetuned classifier (ablation) | H-ensemble · Office-Home 8-shot | accuracy: 0.6571 -> 0.6940 | Helped |
Surprising findings
- Adapted MSFDA methods DECISION and DATE degrade sharply under the few-shot setting, far below trivial baselines.
Limitations
- Requires a linear combination of source feature extractors with equal-dimension features and assumed zero-mean, unit-covariance normalization.
- Evaluation uses a single backbone type (ResNet18) for all source models and task splits constructed from the same datasets.
Experimental setup
Datasets: VisDA-2017 Office-31 Office-Caltech Office-Home
Benchmarks: VisDA-2017 Office-31 Office-Caltech Office-Home
Metrics: accuracy
Baselines: TargetOnly Single-Best Single-Average Average Weights MultiFinetune MCW DECISION DATE
Builds on: maximal correlation regression H-score DECISION DATE
Closest papers by meaning
- A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning (NeurIPS 2025 · similarity 0.88)
- Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learning (NeurIPS 2025 · similarity 0.85)
- Cycle Self-Refinement for Multi-Source Domain Adaptation (AAAI 2024 · similarity 0.85)
- Agile Multi-Source-Free Domain Adaptation (AAAI 2024 · similarity 0.84)
- Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer (CVPR 2024 · similarity 0.84)
- Wasserstein Transfer Learning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.