Research map / Video / Video generation
Animate-X: Universal Character Image Animation with Enhanced Motion Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Animate-X proposes a latent-diffusion character animation framework with an Implicit Pose Indicator (CLIP-based motion features) and Explicit Pose Indicator (pose realignment and rescaling augmentation), plus the A²Bench benchmark of 500 anthropomorphic characters, to animate non-human characters.
Problem
Existing pose-guided character animation methods trained on human dance data fail to generalize to anthropomorphic characters with non-human body structures, forcing rigid pose imposition and identity distortion.
Approach
Enhance motion representation implicitly by querying CLIP image features of the driving video with DWPose-derived and learnable queries, and explicitly by simulating realistic reference-pose misalignments through pose realignment and random body-part rescaling during training.
Key results
- On A2Bench (rescaled pose setting), Animate-X reaches PSNR* 13.60, SSIM 0.452, LPIPS 0.430, FVD 703.87, beating the best baseline Unianimate (FVD 1156.36).
- On TikTok, Animate-X is comparable to Unianimate (FVD 139.01 vs 148.06; SSIM 0.806 vs 0.811), surpassing other diffusion baselines.
- In a 10-participant blind user study, Animate-X was preferred for identity preservation 98.5% of the time versus 43.0% for Unianimate.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the complete Pose Indicator (IPI and EPI) (ablation) | Animate-X · A²Bench rescaled pose | FID-VID: 64.31 -> 32.23 | Helped |
| Add the complete IPI (ablation) | Animate-X · A²Bench rescaled pose | SSIM: 0.433 -> 0.452 | Helped |
| Add the learnable query to IPI (ablation) | Animate-X · A²Bench rescaled pose | SSIM: 0.445 -> 0.452 | Helped |
| Add the DWPose query to IPI (ablation) | Animate-X · A²Bench rescaled pose | FID-VID: 62.34 -> 32.23 | Helped |
| Add EPI (ablation) | Animate-X · A²Bench rescaled pose | SSIM: 0.403 -> 0.452 | Helped |
| Add pose realignment (ablation) | Animate-X EPI · A²Bench rescaled pose | PSNR*: 12.27 -> 13.60 | Helped |
| Add pose rescaling (ablation) | Animate-X EPI · A²Bench rescaled pose | LPIPS: 0.464 -> 0.430 | Helped |
| Set IPI and spatial attention in parallel (ablation) | Animate-X IPI · A²Bench rescaled pose | SSIM: 0.436 -> 0.452 | Helped |
| Use CLIP features as Q and DWPose as K,V in IPI (ablation) | Animate-X IPI · A²Bench rescaled pose | SSIM: 0.443 -> 0.452 | Helped |
| Use a high pose-transformation probability (100% vs 90%) (ablation) | Animate-X EPI · A²Bench | SSIM: 0.444 -> 0.452 | Helped |
| Use a high pose-transformation probability (100% vs 90%) (ablation) | Animate-X EPI · TikTok | SSIM: 0.806 -> 0.802 | Hurt |
| Use Animate-X instead of Unianimate | Character animation · A²Bench rescaled pose | FID: 48.47 -> 26.11 | Helped |
Surprising findings
- Animate-X is trained only on human videos yet generalizes to anthropomorphic characters without limbs.
- Pose realignment contributes more than pose rescaling in the ablation, suggesting simulating misalignment is the key factor.
Limitations
- Insufficient modeling of hands and faces, relying on imperfect DWPose detection.
- Multi-step diffusion makes the method unable to achieve real-time animation.
- A2Bench pose annotations are DWPose-based and only 100 of the 500 videos were manually screened as accurate.
Experimental setup
Datasets: TikTok Fashion approximately 9,000 internet human videos
Benchmarks: A2Bench TikTok Fashion
Metrics: PSNR SSIM L1 LPIPS FID FID-VID FVD
Baselines: Moore-AnimateAnyone MimicMotion ControlNeXt MusePose Unianimate FOMM MRAA LIA DreamPose MagicAnimate
Builds on: Animate Anyone Stable Diffusion Mamba DWPose VideoComposer
Closest papers by meaning
- Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation (CVPR 2024 · similarity 0.91)
- AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Models (CVPR 2025 · similarity 0.87)
- X-Dyna: Expressive Dynamic Human Image Animation (CVPR 2025 · similarity 0.87)
- Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling (ICLR 2025 · similarity 0.86)
- HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.86)
- MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.