Research map / Vision-language and multimodal / Human and embodied modeling
MotionGPT: Human Motion as a Foreign Language
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add instruction tuning to pre-trained MotionGPT Base (ablation) | MotionGPT Base · HumanML3D text-to-motion | FID: 0.365 -> 0.160 | Helped |
| add instruction tuning (ablation) | MotionGPT Base · HumanML3D motion-to-text | Cider: 29.2 -> 28.2 | Hurt |
| add motion-language pre-training before instruction tuning (ablation) | MotionGPT Base · HumanML3D text-to-motion | FID: 0.324 -> 0.160 | Helped |
| add motion-language pre-training before instruction tuning (ablation) | MotionGPT Base · HumanML3D motion prediction | FID: 1.643 -> 0.905 | Helped |
| scale backbone from 60M to 220M (ablation) | MotionGPT · HumanML3D text-to-motion | FID: 0.336 -> 0.160 | Helped |
| scale backbone from 220M to 770M (ablation) | MotionGPT · HumanML3D text-to-motion | FID: 0.160 -> 0.159 | No clear effect |
| scale backbone from 220M to 770M (ablation) | MotionGPT · HumanML3D motion in-between | FID: 0.214 -> 0.223 | Hurt |
| codebook size K=512 instead of 256 (ablation) | VQ-VAE motion tokenizer · HumanML3D reconstruction | FID: 0.187 -> 0.067 | Helped |
| codebook size K=2048 instead of 512 (ablation) | VQ-VAE motion tokenizer · HumanML3D reconstruction | FID: 0.067 -> 0.141 | Hurt |
| task-specific fine-tuning after instruction tuning (ablation) | MotionGPT · HumanML3D text-to-motion | R TOP1: 0.435 -> 0.492 | Helped |
| task-specific fine-tuning after instruction tuning (ablation) | MotionGPT · HumanML3D text-to-motion | FID: 0.160 -> 0.232 | Hurt |
| MotionGPT unified model vs T2M-GPT | MotionGPT · HumanML3D text-to-motion | FID: 0.116 vs 0.232 | Hurt |
Closest papers by meaning
- The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion (CVPR 2025 · similarity 0.92)
- ReMoGPT: Part-Level Retrieval-Augmented Motion-Language Models (AAAI 2025 · similarity 0.92)
- LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning (ICLR 2025 · similarity 0.90)
- Motion-Agent: A Conversational Framework for Human Motion Generation with LLMs (ICLR 2025 · similarity 0.90)
- MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators (AAAI 2024 · similarity 0.90)
- M$^3$GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.