AI Paper Atlas

Research map / Vision-language and multimodal / Visual question answering

Evaluating Text-to-Visual Generation with Image-to-Text Generation

Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, Deva Ramanan

ECCV 2024 · Vision-language and multimodal › Visual question answering · approach: Visual reasoning diagnostics · 282 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use VQAScore (probability of Yes to a VQA question) instead of CLIPScoreimage-text alignment · Winogroundgroup score: 7.8 -> 46.0Improved
use end-to-end VQAScore instead of divide-and-conquer question decomposition with the same VQA model (ablation)CLIP-FlanT5 · Winogroundgroup score: 9.8-16.0 -> 46.0Improved
replace LLaVA-1.5 (Llama-2 decoder-only) with CLIP-FlanT5 (bidirectional encoder-decoder), same training recipe (ablation)VQAScore · Winogroundgroup score: 29.8 -> 46.0Improved
VQAScore with open-source LLaVA-1.5 versus GPT4-Vision methodsimage-text alignment · Winogroundgroup score: LLaVA-1.5 29.8 vs GPT4V-Eval 36.3; CLIP-FlanT5 46.0Mixed
use VQAScore instead of metrics finetuned on human feedbacktext-to-image alignment · Pick-a-Pic (filtered)binary accuracy: 70.0 (PickScore) -> 84.0Improved
apply image-trained VQAScore to videos by averaging frame scoresVQAScore · T2VScore benchmarkpairwise accuracy: 59.9 (CLIPScore) -> 66.5Improved
apply VQAScore to 3D by averaging rendered viewsVQAScore · StanfordT23Dpairwise accuracy: 61.0 (CLIPScore) -> 68.6Improved
divide-and-conquer methods versus end-to-end scoring on DrawBench (ablation)LLaVA-1.5 VQA model · DrawBenchAUROC: Davidsonian 78.8 vs VQAScore LLaVA-1.5 82.2Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy