Research map / 3D vision / Face and shape
TapMo: Shape-aware Motion Generation of Skeleton-free Characters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TapMo, a text-driven pipeline that animates skeleton-free 3D characters by predicting mesh handles and skinning weights and generating handle motion with a shape-aware diffusion model conditioned on mesh deformation features.
Problem
Text-driven motion generation has been limited to pre-rigged SMPL humans; animating non-rigged, heterogeneous 3D characters requires perceiving mesh geometry and lacks ground-truth handles and motions for such characters.
Approach
A GCN-based Mesh Handle Predictor assigns adaptive handles and skinning weights to any mesh and yields a mesh deformation feature; a Transformer-decoder diffusion model conditioned on this feature and CLIP text tokens generates handle motion plus mesh-specific adaptations, trained with weakly-supervised spring, adversarial, and ARAP-based losses.
Key results
- On HumanML3D, TapMo (Ours) reaches FID 0.515 vs 1.058 for TapMo (MDM) and R-Precision Top-1 0.542, outperforming the baselines in R-Precision and FID.
- ARAP-Loss on seen characters is 0.271 for TapMo vs 0.897 for MDM-SR and 1.006 for MDM-SFR (about 70-73% lower).
- In a user study with 100 volunteers, over 84% preferred TapMo animations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TapMo's multi-token and Transformer Decoder diffusion structure instead of the MDM diffusion structure (ablation) | TapMo · HumanML3D | FID: 1.058 -> 0.515 | Helped |
| Use TapMo's multi-token and Transformer Decoder diffusion structure instead of the MDM diffusion structure (ablation) | TapMo · HumanML3D | R-Precision Top-1: 0.498$^{\pm.006}$ -> 0.542$^{\pm.008}$ | Helped |
| Use TapMo's multi-token and Transformer Decoder diffusion structure instead of the MDM diffusion structure (ablation) | TapMo · HumanML3D | Multimodal Distance: 5.126$^{\pm.069}$ -> 5.058$^{\pm.063}$ | Helped |
| Use TapMo's multi-token and Transformer Decoder diffusion structure instead of the MDM diffusion structure (ablation) | TapMo · HumanML3D | Multimodality: 2.744$^{\pm.156}$ -> 2.559$^{\pm.230}$ | Hurt |
| Add mesh-specific motion adaptations (ablation) | TapMo · Seen characters | ARAP-Loss: 0.310 -> 0.271 | Helped |
| Add mesh-specific motion adaptations (ablation) | TapMo · Unseen characters | ARAP-Loss: 0.356 -> 0.329 | Helped |
| Add Spring Loss L_h (ablation) | TapMo · Seen characters | ARAP-Loss: 0.310 -> 0.303 | Helped |
| Add Spring Loss L_h (ablation) | TapMo · Unseen characters | ARAP-Loss: 0.356 -> 0.347 | Helped |
| Add Adversarial Loss L_a (ablation) | TapMo · Seen characters | ARAP-Loss: 0.310 -> 0.307 | No clear effect |
| Add Adversarial Loss L_a (ablation) | TapMo · Unseen characters | ARAP-Loss: 0.356 -> 0.342 | Helped |
| Add motion adaptation fine-tuning (ablation) | TapMo · Seen characters | ARAP-Loss: 0.301 -> 0.271 | Helped |
| Add motion adaptation fine-tuning (ablation) | TapMo · Unseen characters | ARAP-Loss: 0.335 -> 0.329 | Helped |
Surprising findings
- MDM-SR attains the best Handle-FID (0.012), better than TapMo (0.046), because it drives an SMPL skeleton template.
Limitations
- Characters are retrieved from a limited pre-collected model library rather than generated from text.
- Weakly-supervised training on SMPL motions limits generation for quadrupeds or non-human locomotion.
- Fails on characters with non-standard rest poses that deviate from the T-pose used in training.
Experimental setup
Datasets: HumanML3D AMASS Mixamo ModelsResource-RigNet
Benchmarks: HumanML3D ModelsResource-RigNet
Metrics: R-Precision FID Multimodal Distance Diversity Multimodality ARAP-Loss Handle-FID user ranking
Baselines: MDM-SR MDM-SFR T2G Hier TEMOS T2M MDM
Builds on: MDM SfPT CLIP DDPM Linear Blend Skinning
Closest papers by meaning
- DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characters (CVPR 2025 · similarity 0.87)
- HumanRig: Learning Automatic Rigging for Humanoid Character in a Large Scale Dataset (CVPR 2025 · similarity 0.87)
- Make-It-Animatable: An Efficient Framework for Authoring Animation-Ready 3D Characters (CVPR 2025 · similarity 0.87)
- How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects (ICML 2025 · similarity 0.86)
- Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation (CVPR 2024 · similarity 0.86)
- Articulated Kinematics Distillation from Video Diffusion Models (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.