Research map / Vision-language and multimodal / CLIP and zero-shot
Emergent Visual-Semantic Hierarchies in Image-Text Representations
Morris Alper, Hadar Averbuch-Elor
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune with RE contrastive loss on text-only HierarCaps | CLIP ViT-B · HierarCaps test | recall: 0.36 -> 0.47 | Improved |
| fine-tune with RE loss | CLIP ViT-B · HierarCaps test | tau_d: 0.89 -> 0.99 | Improved |
| fine-tune with RE loss | OpenCLIP ViT-H · HyperLex | Spearman rho_all: 0.06 -> 0.37 | Improved |
| fine-tune with RE loss on hierarchy data | CLIP ViT-B · MS-COCO text-to-image | R@1: 0.30 -> 0.31 | No clear effect |
| fine-tune with RE loss | CLIP ViT-B · CIFAR-100 zero-shot | accuracy: 0.66 -> 0.65 | No clear effect |
| use radial embedding contrastive loss instead of entailment cone loss (ablation) | CLIP ViT-B fine-tuning · HierarCaps test | recall: 0.30 -> 0.47 | Improved |
| use pretrained foundation VLM instead of from-scratch hyperbolic MERU | CLIP ViT-L · HierarCaps test | tau_d: MERU 0.79 -> CLIP-L FT 0.97 | Improved |
| use text-only RE fine-tuning compared to dedicated lexical entailment models | CLIP ViT-B · HyperLex | Spearman rho_all: 0.51 vs LEAR 0.69 | Hurt |
Closest papers by meaning
- HieRD: Hierarchical Relational Distillation for Vision-Language Embedding Models (ICML 2026 · similarity 0.91)
- The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models (NeurIPS 2025 · similarity 0.89)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.88)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.88)
- VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.88)
- Language Model as Visual Explainer (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
