Research map / Video / Action recognition
Motion Guided Attention Fusion To Recognize Interactions From Videos
Tae Soo Kim, Jonathan Jones, Gregory D. Hager
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fuse object features with MGAF instead of concatenation (ablation) | I3D single pathway · Something-Else compositional | top-1 accuracy: 54.7 -> 60.5 | Improved |
| add object features via MGAF to single-pathway RGB model | I3D · Something-Else compositional | top-1 accuracy: 46.8 -> 60.5 | Improved |
| fuse objects into the motion pathway with MGAF while keeping appearance pathway | SlowFast dual pathway · Something-Else compositional | top-1 accuracy: 49.6 -> 68.0 | Improved |
| fuse MGAF objects into motion pathway instead of appearance pathway (ablation) | dual pathway + MGAF · Something-Else compositional | top-1 accuracy: 63.8 -> 68.0 | Improved |
| add motion pathway to appearance pathway (RGB only) (ablation) | SlowFast · Something-Else compositional | top-1 accuracy: 46.8 -> 49.6 | Improved |
| replace concatenation with MGAF for dual pathway plus objects (ablation) | dual pathway · Something-Else compositional | top-1 accuracy: 58.8 -> 68.0 | Improved |
| concatenate object features with appearance pathway (ablation) | I3D · Something-Else compositional | top-1 accuracy: 52.3 -> 54.7 | Improved |
| fuse MGAF objects into motion pathway only without appearance pathway (ablation) | motion pathway · Something-Else compositional | top-1 accuracy: 60.5 -> 55.8 | Hurt |
| use larger object-only model v2 over v1 (ablation) | object temporal model · Something-Else compositional | top-1 accuracy: 52.3 -> 53.6 | Improved |
| train and test on disjoint verb-noun compositions rather than mixed | SlowFast · Something-Else compositional | top-1 accuracy: 64.9 -> 49.6 | Hurt |
| use predicted detections instead of ground truth boxes | MGAF(DP, Obj) · Something-Else compositional | top-1 accuracy: 68.0 -> 61.2 | Hurt |
| add MGAF object fusion to dual pathway | SlowFast · IKEA-ASM original task | micro accuracy: 73.5 -> 78.8 | Improved |
Closest papers by meaning
- Attentional Pooling for Action Recognition (NeurIPS 2017 · similarity 0.88)
- Attend and Interact: Higher-Order Object Interactions for Video Understanding (CVPR 2018 · similarity 0.88)
- Interaction Compass: Multi-Label Zero-Shot Learning of Human-Object Interactions via Spatial Relations (ICCV 2021 · similarity 0.88)
- What Have We Learned From Deep Representations for Action Recognition? (CVPR 2018 · similarity 0.87)
- Something-Else: Compositional Action Recognition With Spatial-Temporal Interaction Networks (CVPR 2020 · similarity 0.87)
- CAST: Cross-Attention in Space and Time for Video Action Recognition (NeurIPS 2023 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
