Research map / Vision-language and multimodal / Human and embodied modeling
PersonaBooth: Personalized Text-to-Motion Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Motion Personalization and the PerMo dataset, and PersonaBooth, a multi-modal finetuning of a pretrained motion diffusion model (MDM) with persona tokens, persona cohesion contrastive loss, and context-aware fusion for multiple input motions.
Problem
Generating text-driven motions that reflect a person's persona (unique motion style) from a few atomic reference motions, which existing motion style transfer methods handle only from a single source motion and with limited persona data in pretraining datasets.
Approach
Finetunes a pretrained MDM with two adaptation paths (a learnable persona token injected into the CLIP text encoder and a visual self-attention adapter from a persona extractor), trains the persona extractor with a supervised contrastive persona cohesion loss, and fuses multiple input motions via similarity-weighted top-k context-aware fusion.
Key results
- On PerMo (single input), PersonaBooth reaches FID 3.18 vs 9.16 for MCM-LDM and 13.91 for MoMo, with R-Precision Top-1 0.15 vs 0.13 and 0.05.
- With multiple inputs and CAF on PerMo, FID drops to 2.95 and R-Precision Top-1 rises to 0.19, with PRA avg 0.30 and diversity 0.39.
- On 100Style, PersonaBooth (SI) achieves FID 3.27 vs 5.97 for MoMo and 6.53-7.20 for MCM-LDM variants.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add textual adaptation with P* to visual adaptation (ablation) | PersonaBooth · PerMo, Single Input | FID: 7.45 -> 5.06 | Helped |
| Add persona cohesion loss (ablation) | PersonaBooth · PerMo, Single Input | R-Precision Top 1: 0.05 -> 0.15 | Helped |
| Use multiple input motions rather than one (ablation) | PersonaBooth · PerMo, Multiple Input | FID: 4.27 -> 3.52 | Helped |
| Use CAF instead of averaging all input features (ablation) | PersonaBooth · PerMo, Multiple Input | FID: 3.52 -> 2.95 | Helped |
| Replace self-attention visual adaptation with AdaIN (ablation) | PersonaBooth · PerMo, Single Input | FID: 3.18 -> 8.77 | Hurt |
| Replace self-attention visual adaptation with cross-attention (ablation) | PersonaBooth · PerMo, Single Input | FID: 3.18 -> 8.96 | Hurt |
| Remove text adaptation (ablation) | PersonaBooth · PerMo, Single Input | FID: 3.18 -> 3.61 | Hurt |
| Use PersonaBooth instead of MCM-LDM | Motion generation methods · PerMo | FID: 9.16 -> 3.18 | Helped |
| Use PersonaBooth instead of MCM-LDM* | Motion generation methods · 100Style | FID: 6.53 -> 3.27 | Helped |
Surprising findings
- Adding the mean of multiple input features (InstantBooth-style) gives higher PRA than CAF, yet CAF yields lower FID and higher diversity.
- Finetuning MCM-LDM on PerMo did not improve over the original model on PerMo and 100Style for FID, despite gains in R-precision.
Limitations
- PerMo motion sequences are relatively short, sometimes causing stationary poses at the end of generated motions.
- Context-aware fusion is applied to the whole prompt rather than to each sequential action individually.
- Automatic adjustment of motion length is not addressed.
Experimental setup
Datasets: PerMo 100Style HumanML3D
Benchmarks: PerMo 100Style
Metrics: FID R-Precision Persona Recognition Accuracy Diversity
Baselines: MoMo MCM-LDM MCM-LDM* AdaIN Cross-Attn InstantBooth-style mean fusion
Builds on: MDM InstantBooth TMR MoMo MCM-LDM
Closest papers by meaning
- VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Models (CVPR 2025 · similarity 0.87)
- Visual Persona: Foundation Model for Full-Body Human Customization (CVPR 2025 · similarity 0.86)
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning (ICLR 2024 · similarity 0.86)
- SnapMoGen: Human Motion Generation from Expressive Texts (NeurIPS 2025 · similarity 0.85)
- PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models (CVPR 2024 · similarity 0.85)
- ReMoGPT: Part-Level Retrieval-Augmented Motion-Language Models (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.