Research map / Vision-language and multimodal / Visual question answering
From Images to Textual Prompts: Zero-Shot Visual Question Answering With Frozen Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Generate synthetic image-conditioned QA exemplars rather than use PICa's few-shot exemplars | Frozen-LLM VQA · OK-VQA test | VQA score: 17.7 -> 45.6 | Helped |
| Use Img2LLM instead of Flamingo 80B | VQAv2 test · VQAv2 test | VQA score: 56.3 -> 61.9 | Helped |
| Scale the OPT backbone from 6.7B to 175B | Img2LLM · OK-VQA test | VQA score: 38.2 -> 45.6 | Helped |
| Add synthetic question-answer exemplars to caption prompts (ablation) | Img2LLM · VQAv2 val | VQA score: 46.1 -> 59.5 | Helped |
| Add synthetic question-answer exemplars to caption prompts (ablation) | Img2LLM · OK-VQA | VQA score: 23.5 -> 41.8 | Helped |
| Add captions alongside QA exemplar prompts (ablation) | Img2LLM · VQAv2 val | VQA score: 57.9 -> 59.5 | Helped |
| Add captions alongside QA exemplar prompts (ablation) | Img2LLM · OK-VQA | VQA score: 41.1 -> 41.8 | Helped |
| Use neural-generated questions rather than image-agnostic sampled questions (ablation) | Img2LLM prompt generation · OK-VQA | VQA score: 35.9 -> 40.5 | Helped |
| Select QA pairs by maximum answer frequency rather than randomly (ablation) | Img2LLM prompt generation · OK-VQA | VQA score: 40.5 -> 41.8 | Helped |
| Arrange all captions before all QA pairs rather than interleave caption-QA units (ablation) | Img2LLM prompt · OK-VQA | VQA score: 37.8 -> 41.8 | Helped |
| Select low-frequency answer captions instead of random captions (ablation) | Img2LLM caption selection · OK-VQA | VQA score: 41.3 -> 41.8 | Helped |
| Filter noisy question-relevant captions using image-text similarity (ablation) | Img2LLM · OK-VQA | VQA score: 40.6 -> 41.8 | Helped |
Closest papers by meaning
- Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models (ICLR 2024 · similarity 0.92)
- BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions (AAAI 2024 · similarity 0.90)
- VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning (ICLR 2025 · similarity 0.90)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.90)
- Language Models as Black-Box Optimizers for Vision-Language Models (CVPR 2024 · similarity 0.89)
- CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.