AI Paper Atlas

Research map / Vision-language and multimodal / Visual question answering

Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations

Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, Li Fei-Fei

IJCV 2017 · Vision-language and multimodal › Visual question answering · approach: Vision-language model VQA evaluation · 5,484 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add 5 randomly sampled Visual Genome region descriptions per image instead of one-sentence captions (compared with MS-COCO's 5 captions)Visual Genome region descriptions · Semantic diversity (word2vec clusters per image)clusters per image: 2 clusters in MS-COCO vs 4 or 5 clusters in Visual Genome (t=-240, p<0.01)Improved
collect descriptions region by region instead of image-level captionsVisual Genome · Object densityobjects per image: Visual Genome has 21 objects per image, almost an order of magnitude denser than existing datasetsImproved
restrict Visual Genome to its top 200 categories for comparison with ILSVRC DetectionVisual Genome · Object statistics (top 200 categories only)objects per category: 2239 vs ILSVRC's 2671.5No clear effect
restrict Visual Genome to its top 91 categories for comparison with MS-COCOVisual Genome · Object statistics (top 91 categories only)objects per category: 14.10 -> 3768 objects per categoryImproved
enforce a BLEU score threshold of 0.7 against image-specific and global common descriptionsRegion description collection · Region description diversitydescription diversityImproved
add automatic noun extraction by the dependency parser as worker suggestionsObject extraction crowdsourcing · Object extractionspeed and accuracyImproved
use rapid judgments instead of majority voting for object verificationVerification · Object verificationverification speed: an order of magnitude increase in speedImproved
collect region-based QAs instead of freeform QAsQA collection · QA collectioncost and diversity: around twenty region-based pairs at the same cost of the eight freeform QAsMixed
add hand-crafted WordNet mapping rules plus crowd verification of top four alternative synsetsCanonicalization · Canonicalizationmapping accuracy: 30 hand-crafted noun rules, 15 attribute rules, 20 relationship rulesImproved
collect scene graphs via region descriptions instead of directly from the imageScene graph collection · Relationship annotationrelationship salienceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy