Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Robust Fine-Tuning of Zero-Shot Models
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Interpolate zero-shot and end-to-end fine-tuned weights with α=0.5 | CLIP ViT-L/14@336px · Five ImageNet-derived distribution shifts (average) | accuracy: 4 to 6 percentage points (pp) over prior work | Helped |
| Interpolate zero-shot and end-to-end fine-tuned weights with α=0.5 | CLIP ViT-L/14@336px · ImageNet | accuracy: +1.6 pp | Helped |
| Interpolate zero-shot and end-to-end fine-tuned weights with α=0.5 | CLIP ViT-L/14@336px · Six additional distribution shifts | accuracy: 3.5, 6.2, 1.7, 2.1, 9.0 and 23.2 pp | Helped |
| Interpolate zero-shot and fine-tuned weights | CLIP ViT-L/14@336px · ImageNet | accuracy: 0.8 to 3.3 pp | Helped |
| Interpolate zero-shot and end-to-end fine-tuned weights, α=0.5 | CLIP ViT-L/14@336px · Five ImageNet-derived distribution shifts (average) | accuracy: 68.6 -> 76.9 | Helped |
| Use an optimized mixing coefficient rather than α=0.5 (ablation) | WiSE-FT on CLIP ViT-L/14@336px · Five ImageNet-derived distribution shifts (average) | accuracy: 76.9 -> 77.4 | Helped |
| Use an optimized mixing coefficient rather than α=0.5 (ablation) | WiSE-FT on CLIP ViT-L/14@336px · ImageNet | accuracy: 86.8 -> 87.1 | Helped |
| Increase learning rate from 10⁻⁷ to 3 · 10⁻⁵ during standard fine-tuning (ablation) | CLIP ViT-B/16 · ImageNet | accuracy: 5 pp | Helped |
| Increase learning rate from 10⁻⁷ to 3 · 10⁻⁵ during standard fine-tuning (ablation) | CLIP ViT-B/16 · Five ImageNet-derived distribution shifts (average) | accuracy: 8 pp | Hurt |
| Fine-tune for 10 epochs with learning rate 3 · 10⁻⁵ rather than 3 · 10⁻⁶ (ablation) | CLIP ViT-B/16 · Five ImageNet-derived distribution shifts (average) | accuracy: as much as 8 pp | Mixed |
| Use WiSE-FT with α=0.5 on BASIC-L | BASIC-L · ImageNet | accuracy: 0.4 pp | Helped |
| Use WiSE-FT with α=0.5 on BASIC-L | BASIC-L · Five ImageNet-derived distribution shifts (average) | accuracy: over 7 pp | Helped |
Closest papers by meaning
- Robust Fine-tuning of Zero-shot Models via Variance Reduction (NeurIPS 2024 · similarity 0.90)
- Lipsum-FT: Robust Fine-Tuning of Zero-Shot Models Using Random Text Guidance (ICLR 2024 · similarity 0.90)
- Dual Risk Minimization: Towards Next-Level Robustness in Fine-tuning Zero-Shot Models (NeurIPS 2024 · similarity 0.89)
- Towards Calibrated Robust Fine-Tuning of Vision-Language Models (NeurIPS 2024 · similarity 0.87)
- Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness (CVPR 2024 · similarity 0.87)
- Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.