Research map / Vision-language and multimodal / Cross-modal learning
MMAct: A Large-Scale Dataset for Cross Modal Human Action Understanding
Quan Kong, Ziming Wu, Ziwei Deng, Martin Klinkigt, Bin Tong, Tomokazu Murakami
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add accelerometer teacher via standard distillation (SMD) (ablation) | RGB student · MMAct cross-view | F-measure: 62.21 -> 66.31 | Improved |
| add accelerometer teacher via distillation (SMD) (ablation) | RGB student · MMAct cross-session | F-measure: 69.20 -> 71.23 | Improved |
| add accelerometer teacher via distillation (SMD) (ablation) | RGB student · MMAct cross-subject | F-measure: 64.44 -> 63.89 | No clear effect |
| add gyroscope and orientation teachers (MMD vs SMD) (ablation) | distilled RGB student · MMAct cross-session | F-measure: 71.23 -> 72.08 | Improved |
| add attention weighting over teachers (MMAD vs MMD) (ablation) | multi-teacher distillation · MMAct cross-subject | F-measure: 64.33 -> 66.45 | Improved |
| add attention weighting over teachers (MMAD vs MMD) (ablation) | multi-teacher distillation · MMAct cross-view | F-measure: 68.19 -> 70.33 | Improved |
| add attention weighting over teachers (MMAD vs MMD) (ablation) | multi-teacher distillation · MMAct cross-scene | F-measure: 62.23 -> 64.12 | Improved |
| add attention weighting over teachers (MMAD vs MMD) (ablation) | multi-teacher distillation · MMAct cross-session | F-measure: 72.08 -> 74.58 | Improved |
| distill sensor teachers into RGB student (MMAD) | TSN RGB · MMAct cross-scene | F-measure: 57.91 -> 64.12 | Improved |
| use sensor-only multi-teachers instead of MMAD (ablation) | MMAD · MMAct cross-scene | F-measure: 64.12 -> 67.31 | Mixed |
| distill into RGB student | TSN RGB · MMAct cross-session, carry light | F-measure: 11.12 -> 64.51 | Improved |
| distill into two-stream fusion student (MMAD Fusion) | TSN Fusion · MMAct cross-session | F-measure: 77.09 -> 78.82 | Improved |
Closest papers by meaning
- Interact Before Align: Leveraging Cross-Modal Knowledge for Domain Adaptive Action Recognition (CVPR 2022 · similarity 0.88)
- MMNet: A Model-Based Multimodal Network for Human Action Recognition in RGB-D Videos (TPAMI 2022 · similarity 0.87)
- PULSE: A Synchronized Five-Modality Dataset for Sensorimotor Coordination in Long-Horizon Daily Activities (NeurIPS 2026 · similarity 0.86)
- X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing (ICLR 2025 · similarity 0.86)
- LLAVIDAL: A Large LAnguage VIsion Model for Daily Activities of Living (CVPR 2025 · similarity 0.85)
- LLaVAction: evaluating and training multi-modal large language models for action understanding (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
