Research map / Large language models / Instruction and fine-tuning
LIFT+: Lightweight Fine-Tuning for Long-Tail Learning
Jiang-Xin Shi, Tong Wei, Yu-Feng Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add structured lightweight fine-tuning (SLF) (ablation) | cosine classifier with LA loss · ImageNet-LT | accuracy: 69.3 -> 74.9 | Improved |
| add semantic-aware initialization (SAI) (ablation) | LIFT+ with SLF · ImageNet-LT | accuracy: 74.9 -> 76.9 | Improved |
| add minimalist data augmentation (MDA) (ablation) | LIFT+ with SLF and SAI · ImageNet-LT | accuracy: 76.9 -> 77.0 | Improved |
| add test-time ensembling (TTE) (ablation) | LIFT+ · ImageNet-LT | accuracy: 77.0 -> 78.3 | Improved |
| add structured lightweight fine-tuning (SLF) (ablation) | cosine classifier with LA loss · Places-LT | accuracy: 43.9 -> 48.7 | Improved |
| add semantic-aware initialization (SAI) (ablation) | LIFT+ with SLF · Places-LT | accuracy: 48.7 -> 51.0 | Improved |
| add test-time ensembling (TTE) (ablation) | LIFT+ · Places-LT | accuracy: 51.5 -> 52.1 | Improved |
| fully fine-tune the model instead of only the classifier | CLIP with balanced LA loss · ImageNet-LT | accuracy: 71.6% vs. 73.6% | Hurt |
| fully fine-tune the model instead of lightweight fine-tuning | CLIP · ImageNet-LT | accuracy: 73.2 -> 77.7 | Hurt |
| use semantic-aware initialization instead of random initialization (ablation) | LIFT+ · ImageNet-LT | accuracy: 76.5 -> 78.3 | Improved |
| use cosine classifier (sigma=25) instead of linear classifier (ablation) | LIFT+ · iNaturalist 2018 | accuracy: 76.7 -> 80.5 | Improved |
| use LA loss instead of cross-entropy (ablation) | LIFT+ · ImageNet-LT | accuracy: 72.1 -> 78.3 | Improved |
Closest papers by meaning
- Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts (ICML 2024 · similarity 0.93)
- LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning (ICML 2025 · similarity 0.90)
- LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views (ICML 2024 · similarity 0.88)
- LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups (NeurIPS 2025 · similarity 0.88)
- Linearization Explains Fine-Tuning in Large Language Models (NeurIPS 2025 · similarity 0.88)
- Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
