Research map / Vision-language and multimodal / Visual question answering
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models
Yonatan Bitton, Nitzan Bitton Guetta, Ron Yosef, Yuval Elovici, Mohit Bansal, Gabriel Stanovsky, Roy Schwartz
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune with supervised binary classifier | CLIP ViT-B/32 · WinoGAViL game, 10&12 candidates | Jaccard score: 42 -> 49 | Improved |
| fine-tune with supervised binary classifier | CLIP ViT-B/32 · WinoGAViL game, 5&6 candidates | Jaccard score: 53 -> 52 | No clear effect |
| increase number of candidates from 5&6 to 10&12 | CLIP-ViL · WinoGAViL game | Jaccard score: 47 -> 15 | Hurt |
| increase number of candidates from 5&6 to 10&12 | CLIP-RN50 · WinoGAViL game | Jaccard score: 50 -> 35 | Hurt |
| increase number of candidates from 5&6 to 10&12 | Humans · WinoGAViL game | Jaccard score: 92 -> 90 | No clear effect |
| use game-collected associations instead of SWOW-based | CLIP-VIT-L/14 · 5&6 candidates | Jaccard score: 74 -> 53 | Hurt |
| replace image candidates with OFA captions and text model | MPNet QA · WinoGAViL game 10&12 | Jaccard score: 47 -> 52 | Mixed |
| use image captions with MPNet QA instead of MPNet | text models · WinoGAViL game 10&12 | Jaccard score: 39 -> 47 | Improved |
| visually non-salient cue instead of visually salient cue | CLIP ViT-B/32 · WinoGAViL game 10&12 | model accuracy: 75 -> 36 | Hurt |
| make both cue and candidates images instead of text (SWOW) | CLIP-ViT-L/14 · SWOW modalities | Jaccard index: 86 -> 65 | Hurt |
| CLIP-RN50 with 3 associations vs 2 for 12 candidates | CLIP-RN50 · WinoGAViL game | model performance: 37 -> 33 | Hurt |
Closest papers by meaning
- GameArena: Evaluating LLM Reasoning through Live Computer Games (ICLR 2025 · similarity 0.87)
- Are Large Vision Language Models Good Game Players? (ICLR 2025 · similarity 0.87)
- GameVerse: Can Vision-Language Models Learn from Video-based Reflection? (ICML 2026 · similarity 0.87)
- MM-OPERA: Benchmarking Open-ended Association Reasoning for Large Vision-Language Models (NeurIPS 2025 · similarity 0.86)
- BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (ICLR 2025 · similarity 0.86)
- Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
