AI Paper Atlas

Research map / Video / Video-language understanding

Learning Situation Hyper-Graphs for Video Question Answering

Aisha Urooj, Hilde Kuehne, Bo Wu, Kim Chheu, Walid Bousselham, Chuang Gan, Niels Lobo, Mubarak Shah

CVPR 2023 · Video › Video-language understanding · approach: Video question answering · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Predict action and relationship situation hyper-graphs and use them for question reasoningVideo question answering · AGQA 2.0 Balanced testaccuracy (all): 42.11 vs. 49.20Improved
Predict situation hyper-graphs and reason over them for video QAVideo question answering · STAR testaccuracy (Overall): 36.70 -> 39.47Improved
Use ground-truth rather than predicted hyper-graphs (ablation)SHG-VQA · STAR validationaccuracy (Overall): 40.26 -> 62.46Improved
Add video embeddings to question and hyper-graph cross-attention inputs (ablation)SHG-VQA · STAR testaccuracy (Overall): 39.47 -> 39.18No clear effect
Use both action and relationship tokens rather than action tokens only (ablation)SHG-VQA · STAR validation (single model)accuracy (Overall): 38.68 -> 39.20Improved
Use both action and relationship tokens rather than relationship tokens only (ablation)SHG-VQA · STAR validation (single model)accuracy (Overall): 35.16 -> 39.20Improved
Increase relation queries from 8 to 12 (ablation)SHG-VQA · STAR validation (single model)accuracy (Overall): 39.20 -> 39.90Improved
Increase action queries from 3 to 4 with 8 relation queries (ablation)SHG-VQA · STAR validation (single model)accuracy (Overall): 39.20 -> 37.06Hurt
Reduce action queries from 3 to 2 with 8 relation queries (ablation)SHG-VQA · STAR validation (single model)accuracy (Overall): 39.20 -> 38.34Hurt
Perform set matching frame by frame rather than over the full video (ablation)SHG-VQA · STAR testaccuracy (Overall): 35.15 -> 35.63Mixed
Train with 100K rather than 1.4M QA pairs (ablation)SHG-VQA · AGQA testaccuracy (all): 49.20 -> 43.69Hurt
Remove visual and hyper-graph inputs, retaining question-only information (ablation)SHG-VQA · AGQA testaccuracy (all): 49.20 -> 40.95Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy