Research map / Vision-language and multimodal / Human and embodied modeling
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate vision-language models using twin captions with identical words in a different order | vision-language models on Winoground · Winoground | group score: 16.67 chance; all models below random chance | Hurt |
| Use human crowdworkers instead of tested vision-language models | Winoground · Winoground | group score: 16.67 -> 85.50 | Helped |
| Use FLAVA-ITM rather than the other tested models for image matching | Winoground · Winoground | image score: 20.50 (highest model score) | Helped |
| Increase multimodal pretraining image data | models excluding CLIP and FLAVA · Winoground | text score: 0.84 correlation | Helped |
| Increase multimodal pretraining caption data | models excluding CLIP and FLAVA · Winoground | text score: 0.77 correlation | Helped |
| Use longer captions | best-performing vision-language models · Winoground | group score: -0.20 to -0.23 correlation | Hurt |
| Use longer captions | MTurk Human · Winoground | group score: 0.20 correlation | Helped |
| Use image-text matching rather than contrastive FLAVA configuration | FLAVA · Winoground | image score: 13.50 -> 20.50 | Helped |
| Evaluate on examples with two main predicates rather than one | vision-language models · Winoground, 2 Main Preds vs 1 Main Pred | group score | Hurt |
| Use images from the same photo series | vision-language models · Winoground, Same Image Series | group score: most get 0% | Hurt |
Closest papers by meaning
- Natural Language Inference Improves Compositionality in Vision-Language Models (ICLR 2025 · similarity 0.90)
- Beyond Bag-of-Words: Diagnosing Compositional Binding Failures in Vision-Language Models (NeurIPS 2026 · similarity 0.90)
- Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understanding (CVPR 2024 · similarity 0.89)
- ConMe: Rethinking Evaluation of Compositional Reasoning for Modern VLMs (NeurIPS 2024 · similarity 0.89)
- VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models (ICLR 2026 · similarity 0.89)
- Enhancing Compositional Reasoning in CLIP via Reconstruction and Alignment of Text Descriptions (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.