Research map / Vision-language and multimodal / Visual question answering
Prompting Large Language Models With Answer Heuristics for Knowledge-Based Visual Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Prompt GPT-3 with answer candidates and answer-aware examples from a vanilla VQA model | GPT-3 for knowledge-based VQA · OK-VQA test | accuracy: 53.04 vs. 60.84 | Helped |
| Replace retrieval-based knowledge acquisition with the prompting-based paradigm (ablation) | ViLBERT answer-candidate model · OK-VQA test | accuracy: 40.28 vs. 44.97 | Helped |
| Add answer candidates to the prompt (ablation) | Prophet · OK-VQA test | accuracy: 49.63 -> 56.04 | Helped |
| Increase the answer-candidate list from 1 to 10 (ablation) | Prophet · OK-VQA test | accuracy: 56.04 -> 60.84 | Helped |
| Select in-context examples by fused-feature similarity (ablation) | Prophet · OK-VQA test | accuracy: 58.66 -> 60.84 | Helped |
| Add question and image similarity to fused-feature example selection (ablation) | Prophet · OK-VQA test | accuracy: 60.84 -> 60.38 | Hurt |
| Increase in-context examples from 0 to 20 with 1 query (ablation) | Prophet · OK-VQA test | accuracy: 49.97 -> 57.91 | Helped |
| Increase queries from 1 to 5 (ablation) | Prophet with 16 examples · OK-VQA test | accuracy: 57.52 -> 60.84 | Helped |
| Add confidence scores to answer candidates (ablation) | Prophet · OK-VQA test | accuracy: 55.46 vs. 60.84 | Helped |
| Add image captions to the prompt (ablation) | Prophet · OK-VQA test | accuracy: 58.27 vs. 60.84 | Helped |
| Add predicted object tags to the prompt (ablation) | Prophet · OK-VQA test | accuracy: 60.84 -> 60.51 | Hurt |
| Use CLIP-RN50×64 visual features instead of Bottom-Up features (ablation) | MCAN answer-heuristic model · OK-VQA test | accuracy: 46.83 -> 53.04 | Helped |
Closest papers by meaning
- TOA: Task-oriented Active VQA (NeurIPS 2023 · similarity 0.94)
- Separation of Powers: On Segregating Knowledge from Observation in LLM-enabled Knowledge-based Visual Question Answering (CVPR 2025 · similarity 0.94)
- REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering (NeurIPS 2022 · similarity 0.94)
- StaR-KVQA: Structured Reasoning Traces for Implicit-Knowledge Visual Question Answering (CVPR 2026 · similarity 0.93)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.92)
- BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining (AAAI 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.