Research map / Video / Video generation
MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
Yannan He, Garvita Tiwari, Xiaohan Zhang, Pankaj Bora, Tolga Birdal, Jan Eric Lenssen, Gerard Pons-Moll
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CLIP with T5 text encoder in single-token AdaLN conditioning (ablation) | MARDM-style AE model · HumanML3D (MARDM-67) | FID: 0.114 -> 0.077 | Improved |
| add multi-token cross-attention conditioning instead of single-token AdaLN (ablation) | T5 + AE · HumanML3D (MARDM-67) | FID: 0.077 -> 0.051 | Improved |
| add multi-token cross-attention conditioning instead of single-token AdaLN (ablation) | T5 + AE · HumanML3D (MARDM-67) | R-Precision Top-1: 0.508 -> 0.533 | Improved |
| use semantically aligned autoencoder (SAE) instead of vanilla AE (ablation) | CrossAttn + T5 · HumanML3D (MARDM-67) | R-Precision Top-1: 0.533 -> 0.544 | Improved |
| use semantically aligned autoencoder (SAE) instead of vanilla AE (ablation) | CrossAttn + T5 · HumanML3D (MARDM-67) | FID: 0.051 -> 0.066 | Hurt |
| use cosine semantic loss instead of InfoNCE (ablation) | SAE · HumanML3D (MARDM-67) | FID: 0.129 -> 0.062 | Improved |
| lower semantic loss weight lambda_sem from 0.1 to 0.001 (ablation) | SAE with cosine and KL · HumanML3D (MARDM-67) | FID: 0.071 -> 0.066 | Improved |
| add KL divergence to SAE training (lambda_sem 0.001) (ablation) | SAE with cosine · HumanML3D (MARDM-67) | FID / R-Precision Top-1: FID 0.062 -> 0.066; Top-1 0.535 -> 0.544 | Mixed |
| add a 6-layer text adapter before cross-attention (ablation) | MoLingo (VAE) · HumanML3D (MARDM-67) | FID: 0.057 -> 0.049 | Improved |
| use a 3-layer text adapter instead of none (ablation) | MoLingo (VAE) · HumanML3D (MARDM-67) | FID: 0.057 -> 0.060 | Hurt |
| add repetitive class-token filtering during SAE training (ablation) | MoLingo (SAE) · HumanML3D (MARDM-67) | FID: 0.096 -> 0.066 | Improved |
| add repetitive class-token filtering during SAE training (ablation) | MoLingo (SAE) · HumanML3D (MARDM-67) | R-Precision Top-1: 0.524 -> 0.544 | Improved |
Closest papers by meaning
- Priority-Centric Human Motion Generation in Discrete Latent Space (ICCV 2023 · similarity 0.92)
- Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · similarity 0.91)
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.90)
- SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editing (CVPR 2025 · similarity 0.90)
- SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Prediction (CVPR 2025 · similarity 0.90)
- Enhanced Fine-Grained Motion Diffusion for Text-Driven Human Motion Synthesis (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
