Research map / Video / Video-language understanding
Challenges in multimodal gesture recognition
Sergio Escalera, Vassilis Athitsos, Isabelle Guyon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use winning Motion Signature method instead of template-matching baseline | Baseline method 1 · ChaLearn 2011-12 round 1, final set #1 | Levenshtein error rate: 0.6251 -> 0.0996 | Improved |
| use winning method instead of Principal Motion baseline | Baseline method 2 · ChaLearn 2011-12 round 2, final set #1 | Levenshtein error rate: 0.2997 -> 0.0734 | Improved |
| move from round 1 to round 2 of the challenge (stronger baseline and entries) | Alfnie · ChaLearn 2011-12 final set #1 | Levenshtein error rate: 0.0996 -> 0.0734 | Improved |
| add depth channel to RGB (Turtle Tamers vs Pennect) | HOG/HOF HMM method · ChaLearn 2011-12 round 1/2 | Levenshtein error rate: 0.1652 -> 0.1702 | Mixed |
| use audio-based segmentation with audio+skeleton and late fusion (IVA MM) vs sliding-window skeleton-only | Joewan-style skeleton sliding window · ChaLearn 2013 test set | Levenshtein error rate: 0.83772 -> 0.12756 | Improved |
| evaluate on test set instead of validation set | JOEWAN · ChaLearn 2013 | Levenshtein error rate: 0.13653 -> 0.83772 | Hurt |
| evaluate on test set instead of validation set | WWEIGHT · ChaLearn 2013 | Levenshtein error rate: 0.46163 -> 0.15387 | Improved |
| add late fusion of audio and skeleton (WWEIGHT) compared with single-modality skeleton RF | SUMO skeleton-only RF · ChaLearn 2013 test set | Levenshtein error rate: 0.31652 -> 0.15387 | Improved |
Closest papers by meaning
- ModDrop: Adaptive Multi-Modal Gesture Recognition (TPAMI 2016 · similarity 0.87)
- LD-ConGR: A Large RGB-D Video Dataset for Long-Distance Continuous Gesture Recognition (CVPR 2022 · similarity 0.86)
- SocialGesture: Delving into Multi-person Gesture Understanding (CVPR 2025 · similarity 0.84)
- X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing (ICLR 2025 · similarity 0.83)
- Revisiting Noise Resilience Strategies in Gesture Recognition: Short-Term Enhancement in sEMG Analysis (ICML 2025 · similarity 0.83)
- Multi-Modal Hand-to-Mouth Gesture Recognition in Activity-Oriented RGB-Thermal Footage (Student Abstract) (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
