Research map / Video / Video generation
HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces HumanVid, a large-scale dataset of ~20K curated internet human videos plus synthetic UE5/Blender renders with rule-based camera trajectories, and a baseline CamAnimate that adds camera control to an Animate Anyone-style model.
Problem
Human image animation lacks a large, high-quality public dataset for fair benchmarking, and existing methods ignore camera motion, limiting viewpoint control and video stability.
Approach
Combine rule-filtered real internet videos (with SLAM-estimated camera trajectories and 2D pose annotations) with synthetic renders whose camera paths are sampled by a rule-based keyframe system, then train a camera-conditioned human animation baseline (CamAnimate) using Plücker camera embeddings alongside pose and reference-image conditioning.
Key results
- CamAnimate achieves best performance in almost all metrics on TikTok and UBC-Fashion static-camera tests while also handling moving-camera videos.
- In a 20-participant user study, the proposed method obtained 0.44 average points (upper bound 0.5) and 0.73 top-1 preference over prior methods.
- Training Animate Anyone without camera conditioning on HumanVid is reported to perform markedly worse than CamAnimate on moving-camera test sets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a camera encoder to the Animate Anyone architecture (ablation) | HumanVid-trained human video generation models · HumanVid | performance | Helped |
| Train using Internet videos only, rather than the combined Internet and synthetic data strategy (ablation) | CamAnimate · HumanVid | overall performance | Hurt |
| Use both Internet and synthetic videos in both training stages (ablation) | CamAnimate · HumanVid | overall performance | Hurt |
| Add synthetic videos to improve camera control (ablation) | CamAnimate trained with Internet videos · HumanVid | camera control | Helped |
| Use Internet videos for appearance modeling (ablation) | CamAnimate trained with synthetic data · HumanVid | appearance transfer | Helped |
| Train on synthetic data alone (ablation) | Human image animation models · HumanVid | animation effectiveness | Hurt |
| Increase stage-1 effective batch size from 64 to 256 using gradient accumulation | CamAnimate · HumanVid | performance: 8 × 8 = 64 to 4 × 8 × 8 = 256 | Helped |
| Train CamAnimate on HumanVid instead of the compared prior methods' training setups | CamAnimate · TikTok and UBC-Fashion | performance across metrics | Helped |
| Condition generation on camera motion as well as human pose | CamAnimate · moving-camera human video generation | video quality | Helped |
Surprising findings
- Synthetic data alone is insufficient for appearance modeling, producing non-semantic uniform textures, while internet videos alone give poor camera control; a two-stage schedule combining both works best.
- Standard camera metrics (translation and rotation error) could not be computed reliably because SfM fails on videos with both moving humans and moving cameras.
Limitations
- Human and camera annotations for internet videos depend on pose estimation and SLAM, which can introduce noise.
- Synthetic part has limited assets and background scenes compared with real videos, and rendering quality is below professional real footage.
- Camera control accuracy cannot be quantitatively verified with standard camera metrics on moving-human, moving-camera videos.
Experimental setup
Datasets: HumanVid TikTok UBC-Fashion Bedlam Pexels
Benchmarks: TikTok UBC-Fashion HumanVid internet test set
Metrics: PSNR SSIM LPIPS FID FVD
Baselines: Animate Anyone MagicAnimate Champ Animate Anyone without camera condition (HumanVid variant)
Builds on: Animate Anyone CameraCtrl AnimateDiff Stable Diffusion 1.5 ControlNet
Closest papers by meaning
- HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation (CVPR 2025 · similarity 0.91)
- OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation (CVPR 2025 · similarity 0.90)
- BEDLAM2.0: Synthetic humans and cameras in motion (NeurIPS 2025 · similarity 0.89)
- Move-in-2D: 2D-Conditioned Human Motion Generation (CVPR 2025 · similarity 0.89)
- EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation (CVPR 2025 · similarity 0.89)
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.