Research map / Vision-language and multimodal / CLIP and zero-shot
ViCo: Word Embeddings From Visual Co-Occurrences
Tanmay Gupta, Alexander Schwing, Derek Hoiem
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| concatenate ViCo(linear,100) embeddings to GloVe | GloVe · Visual Genome clustering (fine categories) | V-Measure: 0.50 -> 0.61 | Improved |
| concatenate ViCo(linear,100) embeddings to GloVe | GloVe · Visual Genome clustering (coarse categories) | V-Measure: 0.52 -> 0.65 | Improved |
| train ViCo with WordNet synsets and Object-Hypernym co-occurrences (ablation) | GloVe+ViCo(linear,100) · Visual Genome clustering (fine categories) | V-Measure: 0.54 -> 0.61 | Improved |
| train ViCo with WordNet synsets and Object-Hypernym co-occurrences (ablation) | GloVe+ViCo(linear,100) · Visual Genome clustering (coarse categories) | V-Measure: 0.58 -> 0.65 | Improved |
| concatenate random 100-dim vectors to GloVe (ablation) | GloVe · Visual Genome clustering (coarse categories) | V-Measure: 0.52 -> 0.49 | No clear effect |
| use linear 100-dim transformation instead of select 200-dim transformation, combined with GloVe (ablation) | GloVe+ViCo · Visual Genome clustering (fine categories) | V-Measure: 0.57 -> 0.61 | Improved |
| use select instead of linear 100-dim compression (ablation) | GloVe+ViCo · zero-shot-like unseen class prediction | accuracy | Improved |
| concatenate ViCo (linear) to GloVe | GloVe · Discriminative Attributes | avg F1: 63.85 -> 64.46 | Improved |
| concatenate ViCo (linear) to GloVe | GloVe · MS-COCO caption-image retrieval | Recall@1 (Im2Cap): 44.8 -> 46.3 | Improved |
| concatenate ViCo (linear) to GloVe | GloVe · VQA v2.0 test-dev | overall accuracy: 67.5 -> 67.7 | Improved |
| replace GloVe with random vectors | GloVe · Discriminative Attributes | avg F1: 63.85 -> 50.03 | Hurt |
| replace GloVe with random vectors | GloVe · VQA v2.0 test-dev | overall accuracy: 67.5 -> 66.1 | No clear effect |
Closest papers by meaning
- Visual Word2Vec (vis-w2v): Learning Visually Grounded Word Embeddings Using Abstract Scenes (CVPR 2016 · similarity 0.91)
- Unified Visual-Semantic Embeddings: Bridging Vision and Language With Structured Meaning Representations (CVPR 2019 · similarity 0.91)
- Learning Multi-Modal Word Representation Grounded in Visual Context (AAAI 2018 · similarity 0.90)
- Bridging Vision and Language Spaces with Assignment Prediction (ICLR 2024 · similarity 0.90)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.89)
- Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
