Research map / Vision-language and multimodal / Visual question answering
Modality Shifting Attention Network for Multi-Modal Video Question Answering
Junyeong Kim, Minuk Ma, Trung Pham, Kyungsu Kim, Chang D. Yoo
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add action concepts to visual concepts | MSAN with BERT and visual concepts · TVQA test | accuracy: 70.92 -> 71.13; 0.21% performance gain | Improved |
| Use MPN moment localization instead of the full video and subtitle (ablation) | MSAN · TVQA validation | accuracy: 69.89 -> 70.79; 0.90% | Improved |
| Use the ground-truth temporal moment (ablation) | MSAN · TVQA validation | accuracy: 70.79 -> 71.62; +0.83% | Improved |
| Add self-attention to HRN (ablation) | MSAN · TVQA validation | accuracy: 70.21 -> 70.79; -0.58% | Improved |
| Add context-to-context attention to HRN (ablation) | MSAN · TVQA validation | accuracy: 70.47 -> 70.79; -0.32% | Improved |
| Add MIM to MPN (ablation) | MSAN · TVQA validation | accuracy: 70.56 -> 70.79; -0.23% | Improved |
| Add MIM to HRN (ablation) | MSAN · TVQA validation | accuracy: 70.35 -> 70.79; -0.44% | Improved |
| Apply MIM to additive MPN scoring (ablation) | MPN · TVQA | IoU: 0.25 -> 0.29 | Improved |
| Apply MIM to additive MPN scoring (ablation) | MPN · TVQA | Coverage: 0.32 -> 0.52 | Improved |
| Use multiplicative rather than additive MIM (ablation) | MPN · TVQA | IoU: 0.29 -> 0.31 | Improved |
| Use residual rather than multiplicative MIM (ablation) | MPN · TVQA | IoU: 0.31 -> 0.30 | Hurt |
Closest papers by meaning
- Assessing Modality Bias in Video Question Answering Benchmarks with Multimodal Large Language Models (AAAI 2025 · similarity 0.88)
- Multi-Modality Latent Interaction Network for Visual Question Answering (ICCV 2019 · similarity 0.87)
- Progressive Attention Memory Network for Movie Story Question Answering (CVPR 2019 · similarity 0.86)
- Hierarchical Conditional Relation Networks for Multimodal Video Question Answering (IJCV 2021 · similarity 0.86)
- Modularized Self-Reflected Video Reasoner for Multimodal LLM with Application to Video Question Answering (ICML 2025 · similarity 0.84)
- FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
