Research map / Vision-language and multimodal / Visual question answering
Visual Question Generation for Class Acquisition of Unknown Objects
Kohei Uehara, Antonio Tejero-De-Pablos, Yoshitaka Ushiku, Tatsuya Harada
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use softmax entropy instead of least-confident threshold for unknown detection | ResNet152 classifier · Unknown object classification (ILSVRC2012 known vs ILSVRC2010 unknown) | F measure: 0.635 -> 0.654 | Improved |
| use softmax entropy instead of least-confident threshold for unknown detection | CaffeNet classifier · Unknown object classification (ILSVRC2012 known vs ILSVRC2010 unknown) | F measure: 0.522 -> 0.526 | No clear effect |
| use softmax entropy instead of Bendale et al. for unknown detection | VGGNet classifier · Unknown object classification (ILSVRC2012 known vs ILSVRC2010 unknown) | F measure: 0.553 -> 0.602 | Improved |
| use a stronger classifier (CaffeNet to VGGNet to ResNet) | entropy-based unknown detection · Unknown object classification (ILSVRC2012 known vs ILSVRC2010 unknown) | F measure: 0.526 -> 0.602 -> 0.654 | Improved |
| use entropy rather than Bendale et al. for speed | CaffeNet classifier · Unknown classification execution time | time (sec/image): 15.6 -> 0.0400 | Improved |
| use entropy rather than least confident | CaffeNet classifier · Unknown classification execution time | time (sec/image): 0.0365 -> 0.0400 | No clear effect |
| add target region features and hypernym target word to the decoder (vs whole-image CNN + LSTM) | LSTM question generator · Visual Genome question generation (test) | BLEU-4: 0.110 -> 0.175 | Improved |
| add target region features and hypernym target word to the decoder (vs whole-image CNN + LSTM) | LSTM question generator · Visual Genome question generation (test) | METEOR: 0.163 -> 0.197 | Improved |
| generate questions with the proposed decoder instead of retrieval | question generation · Visual Genome question generation (test) | BLEU-1: 0.438 -> 0.518 | Improved |
| add target conditioning to the LSTM decoder | question generation · Human evaluation, naturalness (AMT) | naturalness score | No clear effect |
| add target conditioning to the LSTM decoder | question generation · Human evaluation, relevance to target region (AMT) | relevance score | Improved |
| use top k=3 instead of k=2 classes for the common hypernym target word (ablation) | unknown object VQG pipeline · Human evaluation, relevance (AMT) | average relevance score | Mixed |
Closest papers by meaning
- Learning by Asking Questions for Knowledge-Based Novel Object Recognition (IJCV 2024 · similarity 0.91)
- An Empirical Evaluation of Visual Question Answering for Novel Objects (CVPR 2017 · similarity 0.87)
- VQA With No Questions-Answers Training (CVPR 2020 · similarity 0.87)
- Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge From External Sources (CVPR 2016 · similarity 0.85)
- Category-Guided Visual Question Generation (Student Abstract) (AAAI 2023 · similarity 0.85)
- An Analysis of Visual Question Answering Algorithms (ICCV 2017 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
