Research map / Vision-language and multimodal / Cross-modal learning
CMD: Self-Supervised 3D Action Representation Learning with Cross-Modal Mutual Distillation
Yunyao Mao, Wengang Zhou, Zhenbo Lu, Jiajun Deng, Houqiang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-modal mutual distillation (joint and motion) (ablation) | MoCo v2 baseline, BiGRU · NTU-60 x-sub linear | top-1 accuracy: 76.1 -> 79.8 | Improved |
| add distillation among all three modalities (ablation) | MoCo v2 baseline · NTU-60 x-sub KNN | top-1 accuracy: 63.4 -> 70.6 | Improved |
| make distillation bidirectional instead of joint<-bone only (ablation) | CMD · NTU-60 x-sub linear | top-1 accuracy: 76.5 -> 77.7 | Improved |
| make distillation bidirectional instead of joint<-motion only (ablation) | CMD · NTU-60 x-sub linear | top-1 accuracy: 78.9 -> 79.8 | Improved |
| make distillation bidirectional instead of joint<-motion only (ablation) | CMD · NTU-60 x-sub KNN | top-1 accuracy: 64.8 -> 68.7 | Improved |
| make distillation bidirectional among three modalities (ablation) | CMD · NTU-60 x-sub KNN | top-1 accuracy: 66.5 -> 70.6 | Improved |
| make distillation bidirectional among three modalities (ablation) | CMD · NTU-60 x-sub linear | top-1 accuracy: 78.8 -> 79.4 | Improved |
| increase number of neighbors K beyond 8192 (ablation) | CMD · NTU-60 x-sub linear | top-1 accuracy | No clear effect |
| replace CrosSCLR-B positive mining with CMD | BiGRU encoder · NTU-60 x-sub KNN | top-1 accuracy: 66.1 -> 70.6 | Improved |
| replace CrosSCLR-B positive mining with CMD | BiGRU encoder · PKU-II transfer from NTU-120 | top-1 accuracy: 52.8 -> 57.0 | Improved |
| replace CrosSCLR-B positive mining with CMD | BiGRU encoder · NTU-60 x-sub semi-supervised 1% | top-1 accuracy: 48.6 -> 50.6 | Improved |
| add multi-modal ensembling at test time | CMD · NTU-60 x-sub linear | top-1 accuracy: 79.8 -> 84.1 | Improved |
Closest papers by meaning
- $$\hbox {I}^2$$MD: 3D Action Representation Learning with Inter- and Intra-Modal Mutual Distillation (IJCV 2025 · similarity 0.95)
- Interact Before Align: Leveraging Cross-Modal Knowledge for Domain Adaptive Action Recognition (CVPR 2022 · similarity 0.88)
- 3D Human Action Representation Learning via Cross-View Consistency Pursuit (CVPR 2021 · similarity 0.88)
- Learning an Augmented RGB Representation With Cross-Modal Knowledge Distillation for Action Detection (ICCV 2021 · similarity 0.87)
- XKD: Cross-Modal Knowledge Distillation with Domain Alignment for Video Representation Learning (AAAI 2024 · similarity 0.87)
- Self-Supervised Learning by Cross-Modal Audio-Video Clustering (NeurIPS 2020 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
