Research map / Vision-language and multimodal / Cross-modal learning
Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition With Multimodal Training
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply multimodal training with SSA and focal regularization rather than independent I3D training | I3D unimodal gesture networks · VIVA hand gestures, RGB and Depth | accuracy: 78.25 -> 81.33; 74.46 -> 81.31 | Helped |
| Apply multimodal training with SSA and focal regularization rather than independent I3D training | I3D unimodal gesture networks · EgoGesture, RGB and Depth | accuracy: 90.33 -> 92.48; 89.47 -> 91.96 | Helped |
| Apply multimodal training with SSA and focal regularization rather than independent I3D training | I3D unimodal gesture networks · NVGesture, RGB, Depth, and Optical Flow | accuracy: 78.42 -> 81.33; 82.28 -> 84.85; 83.19 -> 83.40 | Helped |
| Use MTUT-trained unimodal networks for decision-level fusion instead of I3D networks | I3D modality networks · VIVA hand gestures, RGB+Depth | accuracy: 83.10 -> 86.08 | Helped |
| Use MTUT-trained unimodal networks for decision-level fusion instead of I3D networks | I3D modality networks · EgoGesture, RGB+Depth | accuracy: 92.78 -> 93.87 | Helped |
| Include Optical Flow in training while evaluating RGB+Depth fusion | MTUT fusion model · NVGesture, RGB+Depth fusion | accuracy: 85.48 -> 86.10 | Helped |
| Add SSA without focal regularization (ablation) | C3D gesture networks · VIVA hand gestures, RGB | accuracy: 53.05 -> 53.73 | Helped |
| Add SSA without focal regularization (ablation) | C3D gesture networks · VIVA hand gestures, Depth | accuracy: 55.65 -> 54.52 | Hurt |
| Add SSA without focal regularization (ablation) | I3D gesture networks · VIVA hand gestures, RGB | accuracy: 65.72 -> 65.83 | Helped |
| Add SSA without focal regularization (ablation) | I3D gesture networks · VIVA hand gestures, Depth | accuracy: 67.30 -> 66.96 | Hurt |
| Add focal regularization to SSA (ablation) | I3D gesture networks · VIVA hand gestures, RGB and Depth | accuracy: 65.83 -> 68.43; 66.96 -> 71.26 | Helped |
| Use C3D instead of I3D as the backbone, with the complete MTUT objective (ablation) | MTUT · VIVA hand gestures, RGB and Depth | accuracy: 68.43 vs 56.56; 71.26 vs 58.71 | Hurt |
Closest papers by meaning
- Online Detection and Classification of Dynamic Hand Gestures With Recurrent 3D Convolutional Neural Network (CVPR 2016 · similarity 0.90)
- MMTM: Multimodal Transfer Module for CNN Fusion (CVPR 2020 · similarity 0.89)
- Regional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognition (AAAI 2021 · similarity 0.87)
- Learning Adaptive Hidden Layers for Mobile Gesture Recognition (AAAI 2018 · similarity 0.85)
- Hand1000: Generating Realistic Hands from Text with Only 1,000 Images (AAAI 2025 · similarity 0.83)
- UNIALIGN: Scaling Multimodal Alignment within One Unified Model (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.