Research map / Video / Action recognition
On the Utility of 3D Hand Poses for Action Recognition
Md Salman Shamil, Dibyadip Chatterjee, Fadime Sener, Shugao Ma, Angela Yao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multimodal tokenizer, anticipation loss and verb/object losses (ablation) | HandFormer · H2O | action accuracy: 85.95 -> 93.39 | Improved |
| add multimodal tokenizer, anticipation loss and verb/object losses (ablation) | HandFormer · Assembly101 | action accuracy: 38.98 -> 41.06 | Improved |
| add multimodal tokenizer (ablation) | HandFormer · Assembly101 | action accuracy: 38.98 -> 40.19 | Improved |
| add feature anticipation loss on top of multimodal tokenizer (ablation) | HandFormer · Assembly101 | action accuracy: 40.19 -> 40.24 | No clear effect |
| add verb and object supervision losses (ablation) | HandFormer · H2O | action accuracy: 85.95 -> 90.50 | Improved |
| use 3D instead of 2D hand poses (ablation) | HandFormer-B/6 · Assembly101 verb recognition | verb accuracy: 58.92 -> 63.70 | Improved |
| add a single RGB frame to hand poses (ablation) | HandFormer · Assembly101 | action accuracy: 35.46 vs TSM 35.27 | Improved |
| increase the number of RGB frames | HandFormer · Assembly101 | object accuracy | Mixed |
| replace ViT-g/14 with TSM-pretrained ResNet50 frame encoder (RGB-only) (ablation) | HandFormer · Assembly101 | action accuracy: 32.07 -> 35.09 | Improved |
| replace ViT-g/14 with TSM-pretrained ResNet50 frame encoder (with pose) (ablation) | HandFormer · Assembly101 | action accuracy: 41.06 -> 41.99 | No clear effect |
| combine 3D hand pose with a single RGB view instead of multiple views | HandFormer · Assembly101 | action accuracy: 35.27 -> 41.06 | Improved |
| use 2D heatmaps from PoseConv3D instead of trajectory-based encoding | hand poses · Assembly101 verb recognition | verb accuracy: 46.71 vs 58.92 (HandFormer 2D) | Hurt |
Closest papers by meaning
- Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition From Egocentric RGB Videos (CVPR 2023 · similarity 0.92)
- First-Person Hand Action Benchmark With RGB-D Videos and 3D Hand Pose Annotations (CVPR 2018 · similarity 0.92)
- Collaborative Learning for 3D Hand-Object Reconstruction and Compositional Action Recognition from Egocentric RGB Videos Using Superquadrics (AAAI 2025 · similarity 0.89)
- How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions (CVPR 2025 · similarity 0.89)
- HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video (CVPR 2024 · similarity 0.88)
- Keypoint Fusion for RGB-D Based 3D Hand Pose Estimation (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
