Research map / Video / Action recognition
Egocentric Gesture Recognition Using Recurrent 3D Convolutional Neural Networks With Spatiotemporal Transformer Modules
Congqi Cao, Yifan Zhang, Yi Wu, Hanqing Lu, Jian Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add spatial affine transformer (STM) (ablation) | C3D (RGB) · EgoGesture classification (RGB) | accuracy: 0.864 -> 0.880 | Improved |
| use homography instead of affine transformation in spatial transformer (ablation) | VGG16+LSTM+RSTM (RGB) · EgoGesture classification (RGB) | accuracy: 0.812 -> 0.838 | Improved |
| use homography instead of affine transformation in spatial transformer (ablation) | C3D+STM (RGB) · EgoGesture classification (RGB) | accuracy: 0.880 -> 0.882 | Improved |
| extend spatial transformer to spatiotemporal transformer (STTM) (ablation) | C3D+STM (H) (RGB) · EgoGesture classification (RGB) | accuracy: 0.882 -> 0.887 | Improved |
| add LSTM over the whole gesture sequence (ablation) | C3D (RGB) · EgoGesture classification (RGB) | accuracy: 0.864 -> 0.889 | Improved |
| add STTM to C3D+LSTM (ablation) | C3D+LSTM (RGB) · EgoGesture classification (RGB) | accuracy: 0.889 -> 0.890 | Improved |
| add recurrent connections in the localization network (RSTTM) (ablation) | C3D+LSTM+STTM (RGB) · EgoGesture classification (RGB) | accuracy: 0.890 -> 0.893 | Improved |
| add STTM, then recurrence and LSTM, to C3D on walking scenes | C3D (RGB) · EgoGesture walking scenes (4,6), RGB | accuracy: 0.828 -> 0.839 -> 0.868 | Improved |
| add STTM, then recurrence and LSTM, to C3D on stationary scenes | C3D (RGB) · EgoGesture stationary scenes (1,2,3,5), RGB | accuracy: 0.866 -> 0.870 -> 0.882 | Improved |
| add STTM to C3D with RGB-D sliding-window detection | C3D-l16s8 · EgoGesture detection (RGB-D) | Jaccard index: 0.698 -> 0.709 | Improved |
| add STTM to C3D with depth sliding-window detection | C3D-l16s8 · EgoGesture detection (depth) | Jaccard index: 0.678 -> 0.681 | No clear effect |
| use smaller stride (8 instead of 16) for sliding window | C3D · EgoGesture detection (RGB-D) | Jaccard index: 0.618 -> 0.698 | Improved |
Closest papers by meaning
- First Person Action Recognition Using Deep Learned Descriptors (CVPR 2016 · similarity 0.87)
- Online Detection and Classification of Dynamic Hand Gestures With Recurrent 3D Convolutional Neural Network (CVPR 2016 · similarity 0.87)
- EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activity (NeurIPS 2024 · similarity 0.87)
- H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions (CVPR 2019 · similarity 0.87)
- Egocentric Prediction of Action Target in 3D (CVPR 2022 · similarity 0.86)
- Estimating Ego-Body Pose from Doubly Sparse Egocentric Video Data (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
