Research map / Vision-language and multimodal / Visual question answering
Evaluating Text-to-Visual Generation with Image-to-Text Generation
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, Deva Ramanan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VQAScore (probability of Yes to a VQA question) instead of CLIPScore | image-text alignment · Winoground | group score: 7.8 -> 46.0 | Improved |
| use end-to-end VQAScore instead of divide-and-conquer question decomposition with the same VQA model (ablation) | CLIP-FlanT5 · Winoground | group score: 9.8-16.0 -> 46.0 | Improved |
| replace LLaVA-1.5 (Llama-2 decoder-only) with CLIP-FlanT5 (bidirectional encoder-decoder), same training recipe (ablation) | VQAScore · Winoground | group score: 29.8 -> 46.0 | Improved |
| VQAScore with open-source LLaVA-1.5 versus GPT4-Vision methods | image-text alignment · Winoground | group score: LLaVA-1.5 29.8 vs GPT4V-Eval 36.3; CLIP-FlanT5 46.0 | Mixed |
| use VQAScore instead of metrics finetuned on human feedback | text-to-image alignment · Pick-a-Pic (filtered) | binary accuracy: 70.0 (PickScore) -> 84.0 | Improved |
| apply image-trained VQAScore to videos by averaging frame scores | VQAScore · T2VScore benchmark | pairwise accuracy: 59.9 (CLIPScore) -> 66.5 | Improved |
| apply VQAScore to 3D by averaging rendered views | VQAScore · StanfordT23D | pairwise accuracy: 61.0 (CLIPScore) -> 68.6 | Improved |
| divide-and-conquer methods versus end-to-end scoring on DrawBench (ablation) | LLaVA-1.5 VQA model · DrawBench | AUROC: Davidsonian 78.8 vs VQAScore LLaVA-1.5 82.2 | Mixed |
Closest papers by meaning
- Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation (ICLR 2024 · similarity 0.89)
- Image Content Generation with Causal Reasoning (AAAI 2024 · similarity 0.89)
- LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.89)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.89)
- SK-VQA: Synthetic Knowledge Generation at Scale for Training Context-Augmented Multimodal LLMs (ICML 2025 · similarity 0.88)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
