AI Paper Atlas

Research map / Video / Video-language understanding

KnowIT VQA: Answering Knowledge-Based Questions about Videos

Noa Garcia, Mayu Otani, Chenhui Chu, Yuta Nakashima

AAAI 2020 · Video › Video-language understanding · approach: Video question answering · 78 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add retrieved show-specific knowledge to video, subtitles, questions, and answersROCK · KnowIT VQA test (All)accuracy: 0.587 -> 0.652Improved
Replace retrieved knowledge with annotated ground-truth knowledge (ablation)ROCK · KnowIT VQA test (All)accuracy: 0.652 -> 0.731Improved
Use caption features instead of image features (ablation)ROCK · KnowIT VQA test (All)accuracy: 0.652 -> 0.646Hurt
Use concepts, facial features, or image features in place of one another (ablation)ROCK · KnowIT VQA test (All)accuracy: 0.652No clear effect
Include subtitles with questions and answers (ablation)ROCK_QA · KnowIT VQA test (All)accuracy: 0.530 -> 0.587Improved
Add visual representations to subtitles, questions, and answers (ablation)ROCK_SQA · KnowIT VQA test (All)accuracy: 0.587 -> 0.587No clear effect
Add subtitles using LSTM Embeddings (ablation)LSTM Emb. · KnowIT VQA test (All)accuracy: 0.489 -> 0.467Hurt
Use candidate answers together with the question for knowledge retrieval (ablation)Knowledge retrieval module · KnowIT VQA knowledge retrieval testR@1: 0.070 -> 0.083Improved
Use all candidate answers together, ordered by prior score, rather than one at a time (ablation)Knowledge retrieval module · KnowIT VQA knowledge retrieval testR@1: 0.083 -> 0.114Improved
Provide annotated knowledge to rookies answering questionsRookies answering with subtitles and video · KnowIT VQA testaccuracy: 0.683 -> 0.823Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy