Research map / Large language models / Instruction and fine-tuning
Context-Aware Robust Fine-Tuning
Xiaofeng Mao, Yufeng Chen, Xiaojun Jia, Rong Zhang, Hui Xue, Li Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CAR-FT context KL loss to fine-tuning | CLIP ViT-B/16 FT · ImageNet distribution shifts | avg top-1 accuracy: 45.1 -> 56.9 | Improved |
| use CAR-FT instead of LP-FT | CLIP ViT-B/16 · ImageNet val | top-1 accuracy: 81.7 -> 83.3 | Improved |
| combine CAR-FT with WiSE-FT | WiSE-FT · ImageNet distribution shifts | avg top-1 accuracy: 58.1 -> 58.9 | Improved |
| apply WiSE-FT to CAR-FT | CAR-FT · ImageNet val | top-1 accuracy: 83.3 -> 82.1 | Hurt |
| add CAR-FT to greedy model soups | Greedy Model Soups · ImageNet distribution shifts | avg top-1 accuracy: 56.3 -> 57.1 | Improved |
| add CAR-FT to uniform model soups | Uniform Model Soups · ImageNet distribution shifts | avg top-1 accuracy: 57.5 -> 58.2 | Improved |
| use CAR-FT instead of ERM | CLIP ViT-B/16 · DomainBed | avg accuracy: 72.2 -> 78.5 | Improved |
| use CAR-FT instead of WiSE-FT | CLIP ViT-B/16 · Nine downstream tasks | avg accuracy: 83.8 -> 84.5 | Improved |
| use text-prompt head init (TP-FT) instead of random init (FT) | CLIP ViT-B/16 · Flowers102 | accuracy: 93.7 -> 93.1 | Hurt |
| add CAR-FT on small-data CLIP variants | SLIP-YFCC15M ViT-B/32 · ImageNet distribution shifts | avg top-1 accuracy: 20.7 -> 21.0 | No clear effect |
| reduce context templates from 80 to 7 (CLIP templates) (ablation) | CAR-FT · ImageNet distribution shifts | avg top-1 accuracy: 56.9 -> 56.5 | Hurt |
| use 7 searched templates instead of 7 CLIP templates (ablation) | CAR-FT · ImageNet distribution shifts | avg top-1 accuracy: 56.5 -> 56.9 | Improved |
Closest papers by meaning
- Lipsum-FT: Robust Fine-Tuning of Zero-Shot Models Using Random Text Guidance (ICLR 2024 · similarity 0.89)
- A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation (ICLR 2024 · similarity 0.89)
- Adaptive Parameter Selection for Tuning Vision-Language Models (CVPR 2025 · similarity 0.88)
- Anchor-based Robust Finetuning of Vision-Language Models (CVPR 2024 · similarity 0.87)
- FG-CLIP: Fine-Grained Visual and Textual Alignment (ICML 2025 · similarity 0.87)
- CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
