Research map / Label-efficient and robust learning / Recognition and long-tail
"Idling Neurons, Appropriately Lenient Workload During Fine-tuning Leads to Better Generalization"
Hongjing Niu, Hanting Li, Bin Li, Feng Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add assistant model (AST) and temporary neuron inactivating (TNI) together to fine-tuning (ablation) | ResNet-50 fine-tuning · CIFAR-10 retention after CIFAR-100 fine-tuning | pre-train accuracy: 49.58 -> 59.30 | Improved |
| add AST+TNI (ablation) | ResNet-50 fine-tuning · Unseen CIFAR-100 subset | accuracy: 43.78 -> 51.62 | Improved |
| add AST+TNI (ablation) | ResNet-50 fine-tuning · CIFAR-10.1 | accuracy: 40.52 -> 47.16 | Improved |
| add all three strategies (AST+LPF+TNI) versus a pairwise combination (ablation) | ResNet-50 fine-tuning · CIFAR-10 (Table 2) | accuracy: 60.04 -> 58.05 | Hurt |
| add TNI to fine-tuning | ResNet-50 fine-tuning · CUB-200-2011 (100%) | accuracy: 78.01 -> 79.25 | Improved |
| add AST alone | ResNet-50 fine-tuning · CUB-200-2011 (100%) | accuracy: 78.01 -> 77.42 | No clear effect |
| add LPF alone (ablation) | ResNet-50 fine-tuning · CIFAR-10 retention after CIFAR-100 fine-tuning | pre-train accuracy: 49.58 -> 49.98 | No clear effect |
| apply the strategies to Swin-Transformer-Tiny | Swin-T · ImageNet retention after CIFAR-10 fine-tuning | accuracy: 79.23 -> 82.29 | Improved |
| increase number of CIFAR-10 categories in target task (ablation) | ImageNet-pretrained ResNet-18 · ImageNet retention | accuracy on ImageNet | Hurt |
| increase fine-tuning sample size at fixed categories (ablation) | ImageNet-pretrained ResNet-18 · ImageNet retention | accuracy on ImageNet | No clear effect |
Closest papers by meaning
- LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views (ICML 2024 · similarity 0.88)
- Pay Attention to Small Weights (NeurIPS 2025 · similarity 0.87)
- Domain-Aware Fine-Tuning: Enhancing Neural Network Adaptability (AAAI 2024 · similarity 0.87)
- Fine-Tuning is Fine, if Calibrated (NeurIPS 2024 · similarity 0.87)
- Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting (ICML 2025 · similarity 0.86)
- Mitigating Parameter Interference in Model Merging via Sharpness-Aware Fine-Tuning (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
