Research map / Video / Video generation
MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MimicMotion, a pose-guided human video generator built on Stable Video Diffusion with confidence-aware pose guidance, confidence-weighted hand loss, and training-free progressive latent fusion for arbitrary-length videos.
Problem
Pose-guided human motion video generation suffers from hand distortion, blurred frames, pose estimation noise, and difficulty producing long, temporally smooth videos.
Approach
Encode DWPose keypoint confidence as brightness in the pose guidance map instead of hard thresholding, amplify the diffusion loss in high-confidence hand regions, and fuse overlapping latent segments with position-dependent weights during denoising to produce long smooth videos.
Key results
- On the TikTok test split, MimicMotion reports FID-VID 9.3, FVD 594, SSIM 0.795, PSNR 20.1, the best of the compared methods (best baseline Moore: FID-VID 12.4, FVD 728).
- Ablation: adding confidence-aware pose guiding and hand region enhancement lowers FID-VID from 15.0 to 9.3 and FVD from 678 to 594 relative to progressive fusion alone.
- User study with 36 participants preferred MimicMotion over MuseV at 75.5%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MimicMotion instead of the strongest baseline, Moore | MimicMotion · TikTok test split | FID-VID: 12.4 -> 9.3 | Helped |
| Use MimicMotion instead of the strongest baseline, Moore | MimicMotion · TikTok test split | FVD: 728 -> 594 | Helped |
| Use MimicMotion instead of the strongest baseline, MagicPose | MimicMotion · TikTok test split | SSIM: 0.776 -> 0.795 | Helped |
| Use MimicMotion instead of the strongest baseline, MagicPose | MimicMotion · TikTok test split | PSNR: 18.8 -> 20.1 | Helped |
| Add progressive latent fusion (ablation) | MimicMotion without hand enhancement and confidence-aware pose guidance · Ablation study | FVD: 776 -> 678 | Helped |
| Add progressive latent fusion (ablation) | MimicMotion without hand enhancement and confidence-aware pose guidance · Ablation study | FID-VID: 14.6 -> 15.0 | Hurt |
| Add progressive latent fusion (ablation) | MimicMotion without hand enhancement and confidence-aware pose guidance · Ablation study | SSIM: 0.760 -> 0.758 | Hurt |
| Add progressive latent fusion (ablation) | MimicMotion without hand enhancement and confidence-aware pose guidance · Ablation study | PSNR: 18.0 -> 17.9 | Hurt |
| Add confidence-aware pose guidance (ablation) | MimicMotion with progressive fusion and without hand enhancement · Ablation study | FID-VID: 15.0 -> 12.2 | Helped |
| Add confidence-aware pose guidance (ablation) | MimicMotion with progressive fusion and without hand enhancement · Ablation study | FVD: 678 -> 623 | Helped |
| Add confidence-aware pose guidance (ablation) | MimicMotion with progressive fusion and without hand enhancement · Ablation study | SSIM: 0.758 -> 0.787 | Helped |
| Add confidence-aware pose guidance (ablation) | MimicMotion with progressive fusion and without hand enhancement · Ablation study | PSNR: 17.9 -> 18.4 | Helped |
Surprising findings
- Progressive latent fusion improves FVD but does not clearly improve SSIM or PSNR, which the authors note other metrics may not reflect temporal improvements.
Limitations
- Training data is a small, self-collected set of dancing videos, mostly human dance motion.
- Relies on 2D pose estimation (DWPose), which the authors note has errors such as duplicate detections and front-back ambiguity.
Experimental setup
Datasets: 4,436 internet human dancing videos (self-collected) TikTok dataset (test split, sequences 335-340)
Benchmarks: TikTok test split
Metrics: FID-VID FVD SSIM PSNR user preference
Baselines: MagicAnimate MagicPose Moore-AnimateAnyone MuseV
Builds on: Stable Video Diffusion MultiDiffusion Lumiere DWPose Latent Diffusion Models
Closest papers by meaning
- DiffPerformer: Iterative Learning of Consistent Latent Guidance for Diffusion-based Human Video Generation (CVPR 2024 · similarity 0.88)
- VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models (ICML 2025 · similarity 0.88)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.87)
- Make Pixels Dance: High-Dynamic Video Generation (CVPR 2024 · similarity 0.87)
- TokenMotion: Decoupled Motion Control via Token Disentanglement for Human-centric Video Generation (CVPR 2025 · similarity 0.86)
- HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.