Research map / Vision-language and multimodal / CLIP and zero-shot
Difference Vector Equalization for Robust Fine-tuning of Vision-Language Models
Satoshi Suzuki, Shin'ya Yamaguchi, Shoichiro Takeda, Taiga Yamane, Naoki Makishima, Naotaka Kawata, Mana Ihori, Tomohiro Tanaka, Shota Orihashi, Ryo Masumura
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AVL (average vector loss) to FLYP (ablation) | FLYP · ImageNet OOD avg | top-1 accuracy: 59.5 -> 62.9 | Improved |
| add AVL (average vector loss) to FLYP (ablation) | FLYP · ImageNet zero-shot avg | top-1 accuracy: 49.5 -> 62.9 | Improved |
| add PVL (pairwise vector loss) to FLYP (ablation) | FLYP · ImageNet zero-shot avg | top-1 accuracy: 49.5 -> 62.7 | Improved |
| add PVL on top of AVL (ablation) | DiVE · ImageNet OOD avg | top-1 accuracy: 62.9 -> 63.2 | Improved |
| add PVL on top of AVL (ablation) | DiVE · ImageNet zero-shot avg | top-1 accuracy: 62.9 -> 63.7 | Improved |
| add AVL/PVL to FLYP | FLYP · ImageNet ID | top-1 accuracy: 82.2 -> 82.5 | Improved |
| use EMA average vector (alpha 0.99) instead of per-batch average (alpha 0) (ablation) | AVL · ImageNet zero-shot avg | top-1 accuracy: 61.4 -> 62.9 | Improved |
| constrain difference vectors instead of cosine similarities (ablation) | DiVE · ImageNet zero-shot avg | top-1 accuracy: 61.7 -> 63.7 | Improved |
| use larger reference dataset (CC3M vs Flickr8K) (ablation) | DiVE · ImageNet zero-shot avg | top-1 accuracy: 55.2 -> 63.7 | Improved |
| use DiVE instead of ARF on ImageNet | ARF · ImageNet zero-shot avg | top-1 accuracy: 55.6 -> 63.7 | Improved |
| use DiVE instead of ARF on ImageNet ID | ARF · ImageNet ID | top-1 accuracy: 82.7 -> 82.5 | Hurt |
| allow non-zero difference vectors (DiVE) instead of zero (SnD) | SnD · ImageNet OOD avg | top-1 accuracy: 60.7 -> 63.2 | Improved |
Closest papers by meaning
- VladVA: Discriminative Fine-tuning of LVLMs (CVPR 2025 · similarity 0.89)
- Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization (ICLR 2024 · similarity 0.89)
- Anchor-based Robust Finetuning of Vision-Language Models (CVPR 2024 · similarity 0.89)
- SaCo Loss: Sample-wise Affinity Consistency for Vision-Language Pre-training (CVPR 2024 · similarity 0.88)
- Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models (ICML 2024 · similarity 0.88)
- DH-Set: Improving Vision-Language Alignment with Diverse and Hybrid Set-Embeddings Learning (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
