Research map / Vision-language and multimodal / Visual question answering
Straight to the Facts: Learning Knowledge Base Retrieval for Factual Visual Question Answering
Medhini Narasimhan, Alexander G. Schwing
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add hard negative mining (three iterations) to fact scoring training (ablation) | Ours - Question + Visual Concepts · FVQA supporting fact retrieval | top-1 precision: 20.17 -> 64.5 | Improved |
| add predicted visual concepts as input to the fact-scoring MLP (ablation) | Ours · FVQA answer accuracy | top-1 accuracy: 26.68 -> 60.30 | Improved |
| add predicted visual concepts as input to the fact-scoring MLP (ablation) | Ours · FVQA supporting fact retrieval | top-1 precision: 28.98 -> 62.30 | Improved |
| drop image features, keep question and visual concepts only (ablation) | Ours · FVQA answer accuracy | top-1 accuracy: 60.30 -> 62.20 | Improved |
| drop image features, keep question and visual concepts only (ablation) | Ours · FVQA supporting fact retrieval | top-1 precision: 62.30 -> 64.50 | Improved |
| learned embedding-based relation prediction instead of FVQA baseline | relation prediction · FVQA relation prediction | top-1 accuracy: 64.94 -> 75.4 | Improved |
| learned embedding retrieval instead of FVQA ensemble for answers | answer prediction · FVQA answer accuracy | top-1 accuracy: 58.76 -> 62.20 | Improved |
| learned embedding retrieval instead of FVQA for supporting facts | fact retrieval · FVQA supporting fact retrieval | top-1 precision: 41.12 -> 64.50 | Improved |
| use ground-truth information instead of predicted | Ours - Question + Visual Concepts · FVQA answer accuracy | top-1 accuracy: 62.20 -> 70.34 | Improved |
| second hard negative mining iteration (84,563 hard negatives) (ablation) | Ours · FVQA supporting fact retrieval | top-1 precision: 20.17 -> 38.65 | Improved |
Closest papers by meaning
- Identifying and Resolving Pitfalls of Knowledge-Based VQA Benchmarks: Auditing, Repairing, and Augmenting (ECCV 2026 · similarity 0.90)
- Prompting Large Language Models With Answer Heuristics for Knowledge-Based Visual Question Answering (CVPR 2023 · similarity 0.90)
- FVQA: Fact-Based Visual Question Answering (TPAMI 2017 · similarity 0.90)
- A-OKVQA: A Benchmark for Visual Question Answering Using World Knowledge (ECCV 2022 · similarity 0.90)
- Separation of Powers: On Segregating Knowledge from Observation in LLM-enabled Knowledge-based Visual Question Answering (CVPR 2025 · similarity 0.89)
- Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
