Research map / Video / Action recognition
SUGAR: Learning Skeleton Representation with Visual-Motion Knowledge for Action Recognition
Qilang Ye, Yu Zhou, Lian He, Jie Zhang, Xuanming Guo, Jiayu Zhang, Mingkui Tan, Weicheng Xie, Yue Sun, Tao Tan, Xiaochen Yuan, Ghada Khoriba and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add motion and visual language knowledge to supervise the skeleton encoder (ablation) | SUGAR · Toyota Smarthome overall | accuracy: 69.2 -> 73.4 | Improved |
| add visual knowledge alone (without motion) (ablation) | SUGAR · Toyota Smarthome overall | accuracy: 69.2 -> 69.4 | No clear effect |
| add motion knowledge alone (ablation) | SUGAR · Toyota Smarthome overall | accuracy: 69.2 -> 72.1 | Improved |
| add visual knowledge on top of motion knowledge (ablation) | SUGAR · Toyota Smarthome overall | accuracy: 72.1 -> 73.4 | Improved |
| use TQP instead of a single Q-Former (ablation) | SUGAR · Toyota Smarthome overall | accuracy: 70.7 -> 73.4 | Improved |
| use TQP instead of one linear layer (ablation) | SUGAR · Toyota Smarthome overall | accuracy: 70.4 -> 73.4 | Improved |
| use TQP instead of cross-attention bridging (ablation) | SUGAR · Toyota Smarthome overall | accuracy: 52.1 -> 73.4 | Improved |
| shorten action token sequence from full length 1000 down to moderate lengths (ablation) | SUGAR · NTU RGB+D 60 | accuracy | Improved |
| compress action tokens all the way to 1 (pooling) (ablation) | SUGAR · NTU RGB+D 60 | accuracy | Hurt |
| use SUGAR (LLM recognizer with language-supervised skeleton) instead of LLM-AR | LLM-based recognizer · Toyota Smarthome X-view1 | mean per-class accuracy: 36.1 -> 50.9 | Improved |
| use SUGAR instead of LLM-AR on NTU 60 X-view and NTU 120 X-view | LLM-based recognizer · NTU RGB+D X-view | accuracy: NTU60 X-view 98.4 -> 97.8; NTU120 X-view 91.5 -> 89.7 | Mixed |
| use LLM recognizer (SUGAR) instead of linear classifier for unseen classes | skeleton recognizer · Zero-shot unseen NTU120 classes, Protocol 1 | Top-1 accuracy: 34.5 -> 65.3 | Improved |
Closest papers by meaning
- Heterogeneous Skeleton-Based Action Representation Learning (CVPR 2025 · similarity 0.90)
- Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · similarity 0.90)
- LLMs are Good Action Recognizers (CVPR 2024 · similarity 0.89)
- Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition (CVPR 2025 · similarity 0.89)
- Stitch, Contrast, and Segment: Learning a Human Action Segmentation Model Using Trimmed Skeleton Videos (AAAI 2025 · similarity 0.89)
- Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognition (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
