Research map / Reinforcement learning / RL from feedback
Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the dance accompaniment task with the DD100 MoCap duet dance dataset and Duolando, a GPT follower model with look-ahead conditioning and off-policy RL using step-wise skating rewards.
Problem
Generating a follower's dance motion that responds to a leader's motion and background music in duet dancing, where prior solo or group dance methods cannot condition on a partner and generalization to unseen music/leader motion causes skating artifacts.
Approach
A two-stage pipeline: VQ-VAEs tokenize body parts and relative translation, and an interaction-coordinated GPT with look-ahead transformers predicts follower tokens conditioned on music and leader; an off-policy RL loss, which pushes policy probability toward a sigmoid of step-wise rewards on self-generated OOD samples, reduces skating.
Key results
- Full Duolando reaches FID_k 25.30 and FID_cd 9.97 versus 78.52 and 6643.31 for Bailando adapted without leader conditioning (solo metrics).
- RL reduces skating ratio from 1.06% to 0.33% and improves FID_k from 106.72 to 25.30 relative to the w/o. RL variant.
- In a user study with 15 participants, Duolando is preferred over the w/o. RL tr IC variant 80% of the time, but surpasses ground truth only about 15% of the time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add off-policy RL fine-tuning (skating-aware step rewards) (ablation) | Duolando · DD100 | FID_k: 106.72 -> 25.30 (81.42 lower) | Helped |
| add off-policy RL fine-tuning (ablation) | Duolando · DD100 | skating ratio: 1.06% -> 0.33% | Helped |
| add off-policy RL fine-tuning (ablation) | Duolando · DD100 | FID_cd: 21.68 -> 9.97 | Helped |
| add off-policy RL fine-tuning (ablation) | Duolando · DD100 | CF and BED: CF 57.43 -> 52.36; BED 0.2795 -> 0.2858 | Mixed |
| add relative translation tr VQ-VAE prediction (interaction coordination with translation) (ablation) | Duolando GPT · DD100 | FID_cd: 7970.19 -> 21.68 | Helped |
| add relative translation tr prediction (ablation) | Duolando GPT · DD100 | CF(%): 7.76 -> 57.43 | Helped |
| add relative translation tr prediction (ablation) | Duolando GPT · DD100 | FID_k: 62.29 -> 106.72 | Hurt |
| add leader conditioning and interaction coordination (vs solo, no tr) (ablation) | Duolando GPT · DD100 | BED: 0.1826 -> 0.2194 | Helped |
| add look-ahead mechanism (LAT) (ablation) | Duolando w/o. RL · DD100 | FID_k: 132.73 -> 106.72 | Helped |
| add look-ahead mechanism (LAT) (ablation) | Duolando w/o. RL · DD100 | skating ratio: 1.24% -> 1.06% | Helped |
| add look-ahead mechanism (LAT) (ablation) | Duolando w/o. RL · DD100 | interactive metrics (FID_cd, CF, BED): FID_cd 20.80 vs 21.68; CF 55.81 vs 57.43 | No clear effect |
| randomize music input at test time (ablation) | Duolando · DD100 | BAS: BAS 0.2046 -> 0.2072 (random); BED 0.2858 -> 0.2582 | Mixed |
Surprising findings
- Solo dance baselines get very high FID_cd values (e.g., 6643 for Bailando) because they ignore the leader, and the paper notes that random music barely changes BAS, suggesting BAS is weakly sensitive to music mismatch.
Limitations
- Duolando still falls well short of ground truth on several metrics, and in the user study it beats ground truth only ~15% of the time.
- Explicit relative translation prediction can clash with generated motion and cause skating artifacts without RL.
- Dataset is small (10 pairs of professional dancers, about 117 minutes), and the paper provides no music-information release.
Experimental setup
Datasets: DD100
Benchmarks: DD100
Metrics: FID_k FID_g Div_k Div_g FID_cd Div_cd contact frequency beat echo degree beat align score skating ratio
Baselines: Bailando EDGE Duolando w/o. RL Duolando w/o. RL tr Duolando w/o. RL tr IC
Builds on: Bailando VQ-VAE minGPT actor-critic
Closest papers by meaning
- Breaking the Group Size Barrier: Parameter-Efficient Group Dance Generation with Chain-of-Dancers (NeurIPS 2026 · similarity 0.87)
- DanceAnyWay: Synthesizing Beat-Guided 3D Dances with Randomized Temporal Contrastive Learning (AAAI 2024 · similarity 0.86)
- Bidirectional Autoregessive Diffusion Model for Dance Generation (CVPR 2024 · similarity 0.86)
- Explore 3D Dance Generation via Reward Model from Automatically-Ranked Demonstrations (AAAI 2024 · similarity 0.85)
- Harmonious Music-driven Group Choreography with Trajectory-Controllable Diffusion (AAAI 2025 · similarity 0.85)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.