Research map / Vision-language and multimodal / Visual question answering
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, Li Fei-Fei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 5 randomly sampled Visual Genome region descriptions per image instead of one-sentence captions (compared with MS-COCO's 5 captions) | Visual Genome region descriptions · Semantic diversity (word2vec clusters per image) | clusters per image: 2 clusters in MS-COCO vs 4 or 5 clusters in Visual Genome (t=-240, p<0.01) | Improved |
| collect descriptions region by region instead of image-level captions | Visual Genome · Object density | objects per image: Visual Genome has 21 objects per image, almost an order of magnitude denser than existing datasets | Improved |
| restrict Visual Genome to its top 200 categories for comparison with ILSVRC Detection | Visual Genome · Object statistics (top 200 categories only) | objects per category: 2239 vs ILSVRC's 2671.5 | No clear effect |
| restrict Visual Genome to its top 91 categories for comparison with MS-COCO | Visual Genome · Object statistics (top 91 categories only) | objects per category: 14.10 -> 3768 objects per category | Improved |
| enforce a BLEU score threshold of 0.7 against image-specific and global common descriptions | Region description collection · Region description diversity | description diversity | Improved |
| add automatic noun extraction by the dependency parser as worker suggestions | Object extraction crowdsourcing · Object extraction | speed and accuracy | Improved |
| use rapid judgments instead of majority voting for object verification | Verification · Object verification | verification speed: an order of magnitude increase in speed | Improved |
| collect region-based QAs instead of freeform QAs | QA collection · QA collection | cost and diversity: around twenty region-based pairs at the same cost of the eight freeform QAs | Mixed |
| add hand-crafted WordNet mapping rules plus crowd verification of top four alternative synsets | Canonicalization · Canonicalization | mapping accuracy: 30 hand-crafted noun rules, 15 attribute rules, 20 relationship rules | Improved |
| collect scene graphs via region descriptions instead of directly from the image | Scene graph collection · Relationship annotation | relationship salience | Improved |
Closest papers by meaning
- What's in the Image? A Deep-Dive into the Vision of Vision Language Models (CVPR 2025 · similarity 0.88)
- Synthetic Visual Genome (CVPR 2025 · similarity 0.88)
- PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding (NeurIPS 2025 · similarity 0.88)
- Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models (ICML 2025 · similarity 0.87)
- ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.87)
- VisMin: Visual Minimal-Change Understanding (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
