Research map / Vision-language and multimodal / Cross-modal learning
Multimodal Fusion via Teacher-Student Network for Indoor Action Recognition
Bruce X.B. Yu, Yan Liu, Keith C.C. Chan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add teacher-derived spatial weights to the RGB student network while training both networks (ablation) | ResNet18 student · NTU RGB+D | accuracy: 77.0 -> 79.5 | Improved |
| Add teacher-derived spatial weights to the RGB student network while training both networks (ablation) | ResNet18 student · PKU-MMD | accuracy: 75.2 -> 76.4 | Improved |
| Use teacher-derived spatial weights with the teacher fixed in evaluation mode (ablation) | ResNet18 student · NTU RGB+D | accuracy: 77.0 -> 81.5 | Improved |
| Use teacher-derived spatial weights with the teacher fixed in evaluation mode (ablation) | ResNet18 student · PKU-MMD | accuracy: 75.2 -> 82.5 | Improved |
| Freeze the teacher while training the spatial-weight student rather than updating both networks (ablation) | ResNet18 student · NTU RGB+D | accuracy: 79.5 -> 81.5 | Improved |
| Freeze the teacher while training the spatial-weight student rather than updating both networks (ablation) | ResNet18 student · PKU-MMD | accuracy: 76.4 -> 82.5 | Improved |
| Add the RGB student prediction without spatial weights to the ST-GCN prediction (ablation) | ST-GCN · NTU RGB+D | accuracy: 85.2 -> 91.9 | Improved |
| Add the frozen-teacher spatial-weight student prediction to MS-G3D (ablation) | MS-G3D · NTU RGB+D | accuracy: 92.3 -> 95.0 | Improved |
| Add the frozen-teacher spatial-weight student prediction to MS-G3D (ablation) | MS-G3D · PKU-MMD | accuracy: 95.6 -> 96.8 | Improved |
| Add the frozen-teacher spatial-weight student prediction to AGCN (ablation) | AGCN · NTU RGB+D | accuracy: 86.7 -> 93.0 | Improved |
| Add the frozen-teacher spatial-weight student prediction to AGCN (ablation) | AGCN · PKU-MMD | accuracy: 94.9 -> 96.6 | Improved |
Closest papers by meaning
- MMNet: A Model-Based Multimodal Network for Human Action Recognition in RGB-D Videos (TPAMI 2022 · similarity 0.89)
- Discriminative Multi-Modal Feature Fusion for RGBD Indoor Scene Recognition (CVPR 2016 · similarity 0.85)
- MMAct: A Large-Scale Dataset for Cross Modal Human Action Understanding (ICCV 2019 · similarity 0.85)
- DF2Net: Discriminative Feature Learning and Fusion Network for RGB-D Indoor Scene Classification (AAAI 2018 · similarity 0.85)
- Towards To-a-T Spatio-Temporal Focus for Skeleton-Based Action Recognition (AAAI 2022 · similarity 0.85)
- X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
