Research map / Image generation / Editing and style transfer
Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a diffusion-based multi-condition framework for pose-controlled character image animation with optical flow, depth order, and reference pose guiders to improve background stability and multi-character handling, and introduces a 20-video Multi-Character benchmark.
Problem
Pose-controllable character image animation suffers from unstable backgrounds in noisy training data and from identity confusion and wrong occlusion handling when animating multiple characters.
Approach
Three input guiders (background optical flow with zero-momentum inference, depth-order maps of characters, and reference pose map) are added to a video latent diffusion model to enhance implicit decoupling of background motion, individual characters, and character texture from pose, allowing training on large unfiltered video data.
Key results
- On TikTok, FID-VID 14.30 and FVD 117.81 versus the second-best 21.75 and 153.47 (about 39% and 35% improvement).
- On TED-talks, FID 18.21, LPIPS 0.198, FID-VID 10.24 and FVD 81.73, best on all metrics except PSNR.
- On Multi-Character Bench, FVD 142.76 and FID-VID 14.56, outperforming the second-best by about 36% and 25% respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full multi-condition framework instead of removing all three conditions (ablation) | Ours · TikTok | FID: 54.42 -> 27.70 | Helped |
| Add the optical flow condition (ablation) | Ours · TikTok | FID: 52.94 -> 27.70 | Helped |
| Add the reference pose condition (ablation) | Ours · TikTok | FID: 34.74 -> 27.70 | Helped |
| Add the depth order condition (ablation) | Ours · TikTok | FID: 27.43 -> 27.70 | No clear effect |
Surprising findings
- Reproduced AnimateAnyone trained on the noisy dataset performs much worse than the original, indicating that unfiltered video data weakens baseline models.
- About 88% of the training videos have background optical flow mean above 1, indicating heavy background motion contamination.
Limitations
- Struggles with fine facial and hand details, as is typical for diffusion-based approaches.
- Poor at large-area clothing such as long skirts and Hanfu swaying.
- Fails on scenes with four or more characters or extensive position swapping among characters.
Experimental setup
Datasets: TikTok TED-talks in-house noisy character video set (~2M frames)
Benchmarks: TikTok TED-talks Multi-Character Bench
Metrics: FID SSIM PSNR LPIPS L1 FID-VID FVD
Baselines: MRAA TPSMM DreamPose DisCo DisCo+ MagicAnimate MagicPose AnimateAnyone Moore-AnimateAnyone
Builds on: AnimateAnyone AnimateDiff Stable Diffusion ReferenceNet
Closest papers by meaning
- Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation (CVPR 2024 · similarity 0.87)
- MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling (CVPR 2025 · similarity 0.87)
- Animate-X: Universal Character Image Animation with Enhanced Motion Representation (ICLR 2025 · similarity 0.86)
- EverybodyDance: Bipartite Graph–Based Identity Correspondence for Multi-Character Animation (NeurIPS 2025 · similarity 0.85)
- HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation (CVPR 2025 · similarity 0.85)
- ReMask-Animate: Refined Character Image Animation Using Mask-Guided Adapters (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.