Research map / Label-efficient and robust learning / Self-supervised and contrastive
DanceMVP: Self-Supervised Learning for Multi-Task Primitive-Based Dance Performance Assessment via Transformer Text Prompting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DanceMVP, a contrastively pre-trained ST-GCN+LSTM dance-music encoder with a transformer text prompter for three dance assessment tasks (vocabulary recognition, performance scoring, rhythm evaluation), and releases the ImperialDance dataset.
Problem
Automatic multi-task dance performance assessment on complex, real-world dance motions with music, covering vocabulary recognition, performance scoring, and rhythm (beat alignment) evaluation, where prior AQA work focuses on surgery and sports and single-task scoring.
Approach
Self-supervised InfoNCE pre-training of ST-GCN (motion) and LSTM (music) encoders on eight-beat primitive segments, followed by downstream transformer text prompting (text encoder plus last MLP layer trained, encoders frozen) to perform three assessment tasks, with a new dataset of repeated choreographies across expertise levels.
Key results
- Ours reaches 71.43 accuracy on Task 1 (vocabulary recognition), beating the best baseline CotrastiveDance at 66.85 and the other baselines.
- On Task 3 (rhythm evaluation), the method achieves MSE 2.52, versus 4.20 for SupCon and 5.30 for Dance w/ Melody.
- Prompt tuning outperforms fine-tuning with 5 genres (71.43 vs 66.85 on Task 1), while fine-tuning wins with 1 genre (87.69 vs 86.83).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ST-GCN and LSTM encoders with InfoNCE rather than the reported encoder/loss alternatives (ablation) | DanceMVP · Dance vocabulary recognition | accuracy: 60.68 ± 0.58 -> 71.43 ± 0.71 | Helped |
| Use ST-GCN and LSTM encoders with InfoNCE rather than the reported encoder/loss alternatives (ablation) | DanceMVP · Dance performance scoring | log-likelihood value: 2.56 × 10−4 vs 2.85 × 10−5 | Mixed |
| Use ST-GCN and LSTM encoders with InfoNCE rather than the reported encoder/loss alternatives (ablation) | DanceMVP · Dance rhythm evaluation | MSE loss: 5.51 ± 0.05 -> 2.52 ± 0.73 | Helped |
| Replace text prompt tuning with fine-tuning when five unseen genres are tested (ablation) | DanceMVP · Dance vocabulary recognition | accuracy: 71.43 ± 0.71 -> 66.85 ± 0.19 | Hurt |
| Replace text prompt tuning with fine-tuning when five unseen genres are tested (ablation) | DanceMVP · Dance performance scoring | log-likelihood value: 2.56 × 10−4 -> 0.097 | Hurt |
| Replace text prompt tuning with fine-tuning when five unseen genres are tested (ablation) | DanceMVP · Dance rhythm evaluation | MSE loss: 2.52 ± 0.73 -> 13.65 ± 0.77 | Hurt |
| Replace the LSTM music encoder with ResNet-18 (ablation) | DanceMVP · Dance rhythm evaluation | MSE loss: 2.52 ± 0.73 vs 2.08 ± 0.51 | No clear effect |
Surprising findings
- Fine-tuning beats prompt tuning on Task 1 when only one genre is present (87.69 vs 86.83), so the prompting advantage depends on the number of unseen genres.
- Prompt tuning takes longer to train than fine-tuning because the transformer text encoder is large.
Limitations
- Prompt tuning is slower to train than fine-tuning because the transformer text encoder is large.
- Failure case when dance choreographies are similar.
- Dataset said to be released only in anonymised form, with release described as future; no explicit code or weights release.
Experimental setup
Datasets: ImperialDance AIST++
Benchmarks: ImperialDance AIST++
Metrics: classification accuracy log-likelihood MSE
Baselines: SimCLR MoCo SVT SupCon CotrastiveDance HappyFeet Dance w/ Melody fine-tuning
Builds on: InfoNCE ST-GCN CLIP prompt tuning
Closest papers by meaning
- DanceAnyWay: Synthesizing Beat-Guided 3D Dances with Randomized Temporal Contrastive Learning (AAAI 2024 · similarity 0.86)
- MEGADance: Mixture-of-Experts Architecture for Genre-Aware 3D Dance Generation (NeurIPS 2025 · similarity 0.85)
- Bidirectional Autoregessive Diffusion Model for Dance Generation (CVPR 2024 · similarity 0.85)
- DanceCamera3D: 3D Camera Movement Synthesis with Music and Dance (CVPR 2024 · similarity 0.84)
- DanceFix: An Exploration in Group Dance Neatness Assessment Through Fixing Abnormal Challenges of Human Pose (AAAI 2025 · similarity 0.84)
- Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.