Research map / Large language models / Retrieval and knowledge
VKG-QA: Visual Knowledge Graph-based Question Answer for Large Multimodal Models
Yuntao Du, Yiming Wang, Renshuo Yuan, Jincheng Yue, Yijing Chen, Yue Fan, Bo Zhang, Qian Li, Lizhen Cui
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| present the knowledge graph as an image to an LMM instead of text triples to an LLM | GPT-5 vs GPT-5-chat · VKG-QA text-vs-image subset | average accuracy: 77.1 -> 88.8 | Improved |
| visual input instead of text for connectivity assessment | GPT-5 · VKG-QA connectivity | accuracy: 56.6 -> 94.6 | Improved |
| visual input instead of text for multi-hop and superlative reasoning | LMMs vs LLMs · VKG-QA multi-hop / superlative | accuracy | Hurt |
| increase reasoning depth from 1-hop to 3-hop (ablation) | GPT-5 · VKG-QA hops | accuracy: 91.7 -> 80.2 | Hurt |
| scale Qwen2.5-VL from 7B to 72B (ablation) | Qwen2.5-VL · VKG-QA general image comprehension | accuracy: 62.8 -> 65.3 | Improved |
| scale Qwen2.5-VL from 7B to 32B and 72B (ablation) | Qwen2.5-VL · VKG-QA graph-specific comprehension and reasoning | accuracy | Improved |
| use closed-source rather than open-source LMMs | LMMs · VKG-QA | average accuracy: 85.6 -> 63.1 (best closed vs best open) | Improved |
Closest papers by meaning
- GraphVis: Boosting LLMs with Visual Knowledge Graph Integration (NeurIPS 2024 · similarity 0.92)
- Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models (ICML 2025 · similarity 0.91)
- Multimodal Commonsense Knowledge Distillation for Visual Question Answering (Student Abstract) (AAAI 2025 · similarity 0.90)
- BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining (AAAI 2024 · similarity 0.90)
- Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · similarity 0.90)
- ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
