Research map / Label-efficient and robust learning / Recognition and long-tail
Improved Fine-Tuning by Better Leveraging Pre-Training Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reuse UOT-selected pre-training data during fine-tuning | Supervised ResNet18 · 8-dataset average | top-1 accuracy (%): 78.98 -> 80.15 | Helped |
| Reuse UOT-selected pre-training data during fine-tuning | MoCo-v2 ResNet50 · 8-dataset average, with labels | top-1 accuracy (%): 81.82 -> 84.75 | Helped |
| Reuse UOT-selected pre-training clusters during fine-tuning | MoCo-v2 ResNet50 · 8-dataset average, without labels | top-1 accuracy (%): 81.82 -> 83.17 | Helped |
| Reuse self-supervised pre-training data for Cars | MoCo-v2 ResNet50 · Cars | top-1 accuracy (%): 91.05 -> 90.89 | Hurt |
| Use UOT rather than Greedy-OT selection | Similarity-based selection · CUB, self-supervised with labels | top-1 accuracy (%): 78.63 -> 80.98 | Helped |
| Use UOT rather than Greedy-OT selection | Similarity-based selection · Dogs, self-supervised without labels | top-1 accuracy (%): 81.16 -> 81.47 | Helped |
| Use UOT rather than Greedy-OT selection | Similarity-based selection · Pets, self-supervised without labels | mean per-class accuracy (%): 91.21 -> 90.39 | Hurt |
| Use UOT rather than Greedy-OT selection (ablation) | Similarity-based selection · CUB recall and fine-tuning accuracy | recall rate: 16.95 -> 93.22 | Helped |
| Use cosine distance rather than l2 distance in UOT (ablation) | UOT selection · CUB, self-supervised | top-1 accuracy (%): 78.48 -> 80.98 | Helped |
| Increase the K-means cluster count from 1000 to 2000 (ablation) | Unlabeled-data selection · 8-dataset average | top-1 accuracy (%): 83.04 -> 83.17 | Helped |
| Increase the number of reused pre-training classes (ablation) | UOT fine-tuning on CUB · CUB | top-1 accuracy (%) | Mixed |
| Include more reused pre-training data (ablation) | UOT fine-tuning on Caltech · Caltech | mean per-class accuracy | Helped |
Closest papers by meaning
- Fine-Tuning is Fine, if Calibrated (NeurIPS 2024 · similarity 0.89)
- Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting (ICML 2025 · similarity 0.88)
- LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views (ICML 2024 · similarity 0.88)
- Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How (ICLR 2024 · similarity 0.88)
- Tuning Stable Rank Shrinkage: Aiming at the Overlooked Structural Risk in Fine-tuning (CVPR 2024 · similarity 0.87)
- Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.