Research map / Image generation / Text-to-image diffusion
SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SALAD, a skeleton-aware latent diffusion model using skeleto-temporal VAE and transformer denoiser for text-to-motion generation, plus training-free attention-based zero-shot motion editing via cross-attention modulation.
Problem
Text-driven motion generation typically flattens each pose into a single vector and compresses sentences into one embedding, losing joint-, frame-, and word-level structure; pre-trained motion diffusion models also lack interpretable representations for zero-shot editing.
Approach
Build a skeleto-temporally decoupled motion latent space (skeleton and temporal convolutions and pooling), then train a diffusion denoiser with temporal, skeletal, and word-level cross-attention, and reuse its cross-attention maps for training-free text-driven editing via Prompt-to-Prompt-style modulation.
Key results
- On HumanML3D, SALAD reaches R-precision Top-1 0.581 and Top-3 0.857 versus 0.521 and 0.807 for the best prior baseline MoMask, with FID 0.076.
- On KIT-ML, SALAD achieves Top-1 R-precision 0.477 and FID 0.296, outperforming the baselines in text alignment.
- In a 16-participant user study, SALAD scored 4.596 on preservation and 4.654 on semantic alignment versus 3.729 and 2.758 for MDM, and 3.358 and 3.388 for MotionFix.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SALAD instead of the strongest baseline, MoMask | Text-to-motion generation · HumanML3D test | R-Precision Top-3: 0.807±.002 -> 0.857±.002 | Helped |
| Use SALAD instead of the strongest baseline, MoMask | Text-to-motion generation · KIT-ML test | R-Precision Top-3: 0.781±.005 -> 0.828±.005 | Helped |
| Use SALAD instead of the strongest baseline, MoMask | Text-to-motion generation · HumanML3D test | FID: 0.045±.002 -> 0.076±.002 | Hurt |
| Use SALAD instead of the strongest baseline, ReMoDiffuse | Text-to-motion generation · KIT-ML test | FID: 0.155±.006 -> 0.296±.012 | Hurt |
| Add skeleton-aware ST-Latent instead of a conventional pose-vector VAE (ablation) | SALAD · HumanML3D test | R-Precision Top-3: 0.816±.002 -> 0.857±.002 | Helped |
| Add skeleton-aware ST-Latent instead of a conventional pose-vector VAE (ablation) | SALAD · HumanML3D test | FID: 0.433±.006 -> 0.076±.002 | Helped |
| Add cross-attention instead of sentence-level self-attention (ablation) | SALAD · HumanML3D test | R-Precision Top-3: 0.778±.002 -> 0.857±.002 | Helped |
| Add cross-attention instead of sentence-level self-attention (ablation) | SALAD · HumanML3D test | FID: 0.274±.007 -> 0.076±.002 | Helped |
| Add both ST-Latent and cross-attention (ablation) | SALAD · HumanML3D test | R-Precision Top-3: 0.752±.002 -> 0.857±.002 | Helped |
| Add both ST-Latent and cross-attention (ablation) | SALAD · HumanML3D test | FID: 0.345±.007 -> 0.076±.002 | Helped |
Surprising findings
- The skeleton-aware VAE uses only 0.16M trainable parameters, far fewer than MoMask (19.44M) and ParCo (6.35M), while achieving lower reconstruction FID and MPJPE.
Limitations
- Diversity and MultiModality scores are limited relative to other diffusion methods.
- Restricted to single-person actions with limited text and motion length.
- Editing is demonstrated only on generated motions, not on real motion sequences.
Experimental setup
Datasets: HumanML3D KIT-ML
Benchmarks: HumanML3D KIT-ML
Metrics: FID R-precision MM-Dist Diversity MultiModality MPJPE user preference rating
Baselines: T2M AttT2M ParCo MoMask MDM MLD MotionDiffuse ReMoDiffuse MotionFix
Builds on: Prompt-to-Prompt Skeleton-aware networks DDIM classifier-free guidance T2M evaluation protocol
Closest papers by meaning
- Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · similarity 0.90)
- SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Prediction (CVPR 2025 · similarity 0.89)
- Enhanced Fine-Grained Motion Diffusion for Text-Driven Human Motion Synthesis (AAAI 2024 · similarity 0.88)
- MotionEditor: Editing Video Motion via Content-Aware Diffusion (CVPR 2024 · similarity 0.88)
- AMD: Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion (AAAI 2024 · similarity 0.88)
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.