AI Paper Atlas

Research map / Vision-language and multimodal / Visual question answering

Learning To Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji-Rong Wen, Di Hu

CVPR 2022 · Vision-language and multimodal › Visual question answering · approach: Multimodal question answering datasets · 114 citations (OpenAlex)

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use the proposed spatio-temporal grounding model instead of Pano-AVQAAVQA on MUSIC-AVQA · MUSIC-AVQA testaccuracy (All Avg.): 68.93 -> 71.52Improved
Add audio input to question-only input (ablation)AVQA model · MUSIC-AVQA testaccuracy (All): 54.09 -> 64.26Improved
Add visual input to question-only input (ablation)AVQA model · MUSIC-AVQA testaccuracy (All): 54.09 -> 65.28Improved
Add both audio and visual inputs to question-only input (ablation)AVQA model · MUSIC-AVQA testaccuracy (All): 54.09 -> 67.72Improved
Add temporal grounding (ablation)AV+Q model · MUSIC-AVQA testaccuracy (All): 67.72 -> 70.27Improved
Add spatial grounding (ablation)AV+Q+TG model · MUSIC-AVQA testaccuracy (All): 70.27 -> 71.52Improved
Shuffle video segments in time (ablation)AV+Q+TG · MUSIC-AVQA temporal questionsaccuracy: 65.17 -> 63.71Hurt
Add R(2+1)D motion features (ablation)Our method · MUSIC-AVQAaccuracy: 71.53 -> 71.75Mixed
Add audio input to question-plus-video input (ablation)Our method on TVQA · TVQAaccuracy: 42.01 -> 41.95Hurt
Add video and audio inputs to question-only input (ablation)TVQA method · TVQAaccuracy: 43.50 -> 41.45Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy