Research map / Label-efficient and robust learning / Domain adaptation and generalization
What is being transferred in transfer learning?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize fine-tuning from ImageNet pre-trained weights rather than random initialization | ResNet-50 · DomainNet clipart | top-1 accuracy: 74.32, 57.91 | Helped |
| Initialize fine-tuning from ImageNet pre-trained weights rather than random initialization | ResNet-50 · CheXpert and DomainNet real, clipart, quickdraw | test performance | Helped |
| Shuffle image blocks to increasingly disrupt visual features while preserving pixel-value statistics (ablation) | P-T versus RI-T · DomainNet real and clipart | relative accuracy benefit from transfer | Mixed |
| Shuffle image blocks, including to 1×1 blocks (ablation) | P-T versus RI-T · DomainNet quickdraw | relative accuracy benefit from transfer | No clear effect |
| Use pre-trained initialization instead of random initialization (ablation) | Target-domain training · Block-shuffled images | optimization speed | Helped |
| Use a larger learning rate for fine-tuning rather than the smaller learning rate (ablation) | P-T · CheXpert | convergence | Hurt |
| Use a larger learning rate rather than the smaller learning rate (ablation) | P-T · DomainNet quickdraw | test performance | Hurt |
| Use a larger learning rate rather than the smaller learning rate (ablation) | RI-T · DomainNet real, clipart, quickdraw | test performance | Helped |
| Use an earlier ImageNet pre-training checkpoint to initialize fine-tuning (ablation) | P-T · DomainNet real and clipart | final performance | Mixed |
| Initialize from checkpoints around learning-rate decay rather than the plateauing checkpoints (ablation) | P-T · DomainNet real and clipart | final performance | Hurt |
| Train two RI-T runs from identical random initialization rather than independent random initializations (ablation) | RI-T · DomainNet real, clipart, quickdraw | interpolation performance barrier | Helped |
Closest papers by meaning
- Features are fate: a theory of transfer learning in high-dimensional regression (ICML 2025 · similarity 0.89)
- Adversarially-Trained Deep Nets Transfer Better: Illustration on Image Classification (ICLR 2021 · similarity 0.89)
- A Data-Based Perspective on Transfer Learning (CVPR 2023 · similarity 0.89)
- Transfusion: Understanding Transfer Learning for Medical Imaging (NeurIPS 2019 · similarity 0.88)
- Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model (ICLR 2024 · similarity 0.86)
- Transferring Knowledge From Large Foundation Models to Small Downstream Models (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.