Research map / Vision-language and multimodal / CLIP and zero-shot
LOVM: Language-Only Vision Model Selection
Orr Zohar, Shih-Cheng Huang, Kuan-Chieh Wang, Serena Yeung
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add granularity scores to ImageNet benchmark scores | LOVM ranking · LOVM benchmark | Top-5 Recall (mean per-class recall): 0.504 -> 0.539 | Improved |
| Add granularity scores to ImageNet benchmark scores | LOVM ranking · LOVM benchmark | Kendall's tau (mean per-class recall): 0.186 -> 0.197 | Improved |
| Add text classification scores to ImageNet benchmark scores | LOVM ranking · LOVM benchmark | Kendall's tau (top-1 accuracy): 0.177 -> 0.223 | Improved |
| Use granularity scores instead of ImageNet benchmark scores | LOVM performance prediction · LOVM benchmark | L1 (mean per-class recall): 0.228 -> 0.145 | Improved |
| Use granularity scores instead of ImageNet benchmark scores | LOVM performance prediction · LOVM benchmark | L1 (top-1 accuracy): 0.220 -> 0.143 | Improved |
| Add ImageNet benchmark scores to text classification scores | LOVM performance prediction · LOVM benchmark | L1 (mean per-class recall): 0.182 -> 0.182 | No clear effect |
| Add ImageNet benchmark scores to granularity scores | LOVM performance prediction · LOVM benchmark | L1 (top-1 accuracy): 0.143 -> 0.141 | Improved |
| Inject Gaussian noise into text-derived embeddings (ablation) | Text top-1 accuracy proxy · LOVM benchmark | Correlation with ground-truth top-1 accuracy | Mixed |
Closest papers by meaning
- Vision-Language Model Selection and Reuse for Downstream Adaptation (ICML 2025 · similarity 0.93)
- Bridge the Modality and Capability Gaps in Vision-Language Model Selection (NeurIPS 2024 · similarity 0.92)
- Mordal: Automated Pretrained Model Selection for Vision Language Models (ICLR 2026 · similarity 0.92)
- Are We on the Right Way for Evaluating Large Vision-Language Models? (NeurIPS 2024 · similarity 0.92)
- Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distance (CVPR 2026 · similarity 0.91)
- What matters when building vision-language models? (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
