Research map / Video / Action recognition
Modality Distillation with Multiple Stream Networks for Action Recognition
Nuno C. Garcia, Pietro Morerio, Vittorio Murino
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-stream multiplicative connections when training hallucination stream (ablation) | hallucination network (step 3) · NTU RGB+D (cross-subject) | accuracy: 70.64 -> 71.93 | Improved |
| add cross-stream connections (cross-subject, step 3) (ablation) | hallucination network · NTU RGB+D (cross-subject) | accuracy: 71.10 -> 71.93 | Improved |
| add cross-stream connections to the two-stream RGB+D teacher (ablation) | step 2 model · NTU RGB+D (cross-subject) | accuracy: 78.27 -> 79.73 | Improved |
| add cross-stream connections to the two-stream RGB+D teacher (ablation) | step 2 model · NTU RGB+D (cross-view) | accuracy: 82.11 -> 81.43 | Hurt |
| replace Euclidean loss with distillation loss L_GD in Hoffman pipeline (ablation) | Hoffman et al. hallucination · NTU RGB+D (cross-subject) | accuracy: 64.64 -> 68.60 | Improved |
| use combined loss L instead of L_GD in Hoffman pipeline (ablation) | Hoffman et al. hallucination · NTU RGB+D (cross-subject) | accuracy: 68.60 -> 70.70 | Improved |
| use proposed losses and staged training instead of original Hoffman setup (ablation) | Hoffman et al. · NTU RGB+D (cross-subject) | accuracy: 64.64 -> 70.70 | Improved |
| add staged training with fine-tuning (step 4) after hallucination learning (ablation) | step 3 model · NTU RGB+D (cross-subject) | accuracy: 71.93 -> 73.42 | Improved |
| add fine-tuning step 4 (ablation) | step 3 model · NTU RGB+D (cross-view) | accuracy: 74.10 -> 77.21 | Improved |
| hallucinate depth instead of training RGB-only (ablation) | RGB stream · NTU RGB+D (cross-subject) | accuracy: 66.52 -> 73.42 | Improved |
| use hallucination model instead of corrupted real depth input | two-stream model · NTU RGB+D (cross-view) | accuracy: 76.85 -> 77.21 | Improved |
| increase depth noise variance from 10^-3 to 10^1 | two-stream RGB+D with real depth · NTU RGB+D (cross-view) | accuracy: 81.34 -> 51.43 | Hurt |
Closest papers by meaning
- Graph Distillation for Action Detection with Privileged Modalities (ECCV 2018 · similarity 0.91)
- Learning with Privileged Information via Adversarial Discriminative Modality Distillation (TPAMI 2019 · similarity 0.90)
- Learning an Augmented RGB Representation With Cross-Modal Knowledge Distillation for Action Detection (ICCV 2021 · similarity 0.90)
- Local-Global Multi-Modal Distillation for Weakly-Supervised Temporal Video Grounding (AAAI 2024 · similarity 0.86)
- XKD: Cross-Modal Knowledge Distillation with Domain Alignment for Video Representation Learning (AAAI 2024 · similarity 0.86)
- DVANet: Disentangling View and Action Features for Multi-View Action Recognition (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
