Research map / Video / Video generation
AniMo: Species-Aware Model for Text-Driven Animal Motion Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AniMo, a two-stage (RVQ motion tokenizer + masked-modeling text-to-motion transformer) model for text-driven animal motion generation, and introduces the AniMo4D dataset of 78,149 motions across 114 species.
Problem
Text-to-motion generation has focused on humans; generating animal motion from text is under-explored due to species diversity, varied morphology, and limited data.
Approach
A joint-aware spatiotemporal encoder with a skeleton-tree attention mask (limb-restricted spatial transformer followed by TCN), plus species-aware feature-wise linear modulation driven by CLIP embeddings of species and sex, feeding a masked-modeling base/residual transformer over RVQ tokens.
Key results
- On AniMo4D, AniMo reaches Top-1 R-Precision 0.774 and FID 0.908 versus 0.708 and 0.870 for the next-best MoMask-style baseline in the table (MoMask Top-1 0.708, FID 0.870).
- On the out-of-distribution AnimalML3D set, AniMo achieves Top-1 R-Precision 0.144 and FID 0.254, the best among compared methods on those metrics.
- Ablation on AniMo4D shows removing SAFM raises generation FID from 0.037 to 0.056 and removing S-Trans raises it to 0.071.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AniMo instead of MoMask | AniMo text-driven animal motion generation · AniMo4D test | FID: 0.143±.000 -> 0.029±.000 | Helped |
| Use AniMo instead of MoMask | AniMo text-driven animal motion generation · AniMo4D test | R-Precision Top-1: 0.708±.002 -> 0.774±.002 | Helped |
| Use AniMo instead of T2M-GPT | AniMo text-driven animal motion generation · AniMo4D test | Multimodality: 0.519±.018 -> 0.368±.015 | Hurt |
| Use AniMo instead of T2M-GPT | AniMo text-driven animal motion generation · AnimalML3D OOD | FID: 2.843±.085 -> 2.710±.086 | Helped |
| Use AniMo instead of MoMask | AniMo text-driven animal motion generation · AnimalML3D OOD | MM-Dist: 3.970±.016 -> 3.867±.017 | Helped |
| Use AniMo instead of T2M-GPT | AniMo text-driven animal motion generation · AnimalML3D OOD | Multimodality: 0.947±.038 -> 0.947±.042 | No clear effect |
| Add Species-Aware Feature Modulation (SAFM) (ablation) | AniMo · AniMo4D generation | FID: 0.056 -> 0.037 | Helped |
| Add Spatial Transformer (S-Trans) (ablation) | AniMo · AniMo4D generation | FID: 0.071 -> 0.037 | Helped |
| Add discriminator D(·) (ablation) | AniMo · AniMo4D reconstruction | MPJPE: 0.325 -> 0.206 | Helped |
Surprising findings
- Human text-to-motion baselines retrained on animal skeletons are reported to degrade substantially on the OOD AnimalML3D set, though AniMo still leads on several metrics.
- AniMo has a lower MModality than some baselines, which the authors attribute to prioritizing text adherence over diversity.
Limitations
- No publicly available pure text-driven animal motion baselines; baselines are human models retrained with modified skeletons.
- AniMo4D is derived from a simulation game (Planet Zoo), so it is synthetic-domain data and some species (e.g., penguins) were excluded.
- Lower MultiModality than some baselines indicates a diversity trade-off.
Experimental setup
Datasets: AniMo4D AnimalML3D
Benchmarks: AniMo4D AnimalML3D
Metrics: FID R-Precision MM-Dist Diversity MultiModality MPJPE
Baselines: T2M MDM T2M-GPT AttT2M MMM MoMask OmniMotionGPT
Builds on: MoMask T2M T2M-GPT SinMDM
Closest papers by meaning
- OmniMotionGPT: Animal Motion Generation with Limited Data (CVPR 2024 · similarity 0.89)
- How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects (ICML 2025 · similarity 0.88)
- Animal-Bench: Benchmarking Multimodal Video Models for Animal-centric Video Understanding (NeurIPS 2024 · similarity 0.86)
- Web-Scale Collection of Video Data for 4D Animal Reconstruction (NeurIPS 2025 · similarity 0.84)
- Learning the 3D Fauna of the Web (CVPR 2024 · similarity 0.84)
- BEAST3D: animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.