Research map / Video / Action recognition
Multi-Modal Multi-Action Video Recognition
Zhensheng Shi, Ju Liang, Qianqian Li, Haiyong Zheng, Zhaorui Gu, Junyu Dong, Bing Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace fully-connected classifier with visual GCN (ablation) | R(2+1)D-34 · Mini M-MiT | mAP: 54.8 -> 55.0 | Improved |
| replace fully-connected classifier with visual GCN (ablation) | R(2+1)D-34 · Mini M-MiT | top-1 accuracy: 52.1 -> 53.3 | Improved |
| add audio GCN to visual model (ablation) | J(H, Gv) · Mini M-MiT | mAP: 55.0 -> 58.0 | Improved |
| add textual GCN to visual model (ablation) | J(H, Gv) · Mini M-MiT | mAP: 55.0 -> 58.2 | Improved |
| add visual GCN to audio GCN joint model (ablation) | J(H, Ga) · Mini M-MiT | mAP: 58.0 -> 58.2 | Improved |
| add visual GCN to textual GCN joint model (ablation) | J(H, Gt) · Mini M-MiT | mAP: 58.2 -> 58.5 | Improved |
| add textual GCN to visual plus audio GCN (all three modalities) (ablation) | J(H, Gv, Ga) · Mini M-MiT | mAP: 58.2 -> 58.7 | Improved |
| add third modality over two modalities (ablation) | J(H, Gv, Ga, Gt) · Mini M-MiT | top-1 accuracy: 55.1 -> 55.0 | No clear effect |
| use 2 synonymous audio features per action instead of 1 (ablation) | J(H, Ga) · Mini M-MiT | mAP: 58.0 -> 57.8 | Hurt |
| use 3 synonymous audio features per action instead of 1 (ablation) | J(H, Ga) · Mini M-MiT | mAP: 58.0 -> 57.4 | Hurt |
| BERT 768D instead of GloVe 300D text lexicon (ablation) | J(H, Gt) · Mini M-MiT | mAP: 58.2 -> 58.2 | No clear effect |
| merge audio and textual lexicons into one audio-textual GCN (ablation) | J(H, Ga,t) · Mini M-MiT | mAP: 58.0 -> 58.4 | Improved |
Closest papers by meaning
- Multi-view Action Recognition using Cross-view Video Prediction (ECCV 2020 · similarity 0.89)
- MMNet: A Model-Based Multimodal Network for Human Action Recognition in RGB-D Videos (TPAMI 2022 · similarity 0.88)
- Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (TPAMI 2017 · similarity 0.88)
- DVANet: Disentangling View and Action Features for Multi-View Action Recognition (AAAI 2024 · similarity 0.87)
- Open-Vocabulary Video Relation Extraction (AAAI 2024 · similarity 0.87)
- Prompt-guided Disentangled Representation for Action Recognition (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
