Research map / Vision-language and multimodal / Visual question answering
Beyond RNNs: Positional Self-Attention with Co-Attention for Video Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| PSAC with ResNet only vs Co-memory(R+F) | video QA · TGIF-QA Action | accuracy: 68.2 -> 70.4 | Helped |
| PSAC with ResNet only vs Co-memory(R+F) | video QA · TGIF-QA Count | MSE: 4.10 -> 4.27 | Hurt |
| add video positional self-attention (VPSA) (ablation) | PSAC · TGIF-QA Action | accuracy: 69.0 -> 70.4 | Helped |
| add video positional self-attention (VPSA) (ablation) | PSAC · TGIF-QA FrameQA | accuracy: 55.5 -> 55.7 | No clear effect |
| stack more VPSA layers (1 -> 5) (ablation) | PSAC · TGIF-QA Action | accuracy: 70.4 -> 64.6 | Hurt |
| VPSA instead of Bi-LSTM (ablation) | PSAC · TGIF-QA Action | accuracy: 67.6 -> 70.4 | Helped |
| VPSA instead of Bi-LSTM (ablation) | PSAC · TGIF-QA Action | training time s/epoch: 337 -> 269 | Helped |
| add character embeddings to question encoding (ablation) | PSAC · TGIF-QA Action | accuracy: 66.89 -> 70.36 | Helped |
| add video-question co-attention (ablation) | PSAC · TGIF-QA Action | accuracy: 67.15 -> 70.36 | Helped |
| add video-question co-attention (ablation) | PSAC · TGIF-QA FrameQA | accuracy: 55.84 -> 55.67 | No clear effect |
Closest papers by meaning
- Structured Two-Stream Attention Network for Video Question Answering (AAAI 2019 · similarity 0.89)
- Hierarchical Question-Image Co-Attention for Visual Question Answering (NeurIPS 2016 · similarity 0.88)
- Language-aware Visual Semantic Distillation for Video Question Answering (CVPR 2024 · similarity 0.88)
- Deep Modular Co-Attention Networks for Visual Question Answering (CVPR 2019 · similarity 0.88)
- Self-Chained Image-Language Model for Video Localization and Question Answering (NeurIPS 2023 · similarity 0.88)
- STAIR: Spatial-Temporal Reasoning with Auditable Intermediate Results for Video Question Answering (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.