Research map / Image generation / Editing and style transfer
ReMask-Animate: Refined Character Image Animation Using Mask-Guided Adapters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReMask-Animate, a diffusion-based human image animation framework with three mask-guided adapters (P, S, C) that guide attention to hand/face, foreground/background, and identity regions, trained on TikTok and Fashion.
Problem
Pose-controlled human image animation suffers from regional blurring, background sharpening/feature confusion between foreground and background, and weak identity consistency.
Approach
Uses masks (hand/face, foreground/background) as priors inside lightweight adapters: a gated mask-guided P-Adapter fuses DWPose and SMPL pose features regionally, a mask-guided S-Adapter separates foreground and background self-attention in the decoder, and a mask-guided C-Adapter injects CLIP facial features into foreground regions.
Key results
- On TikTok, ReMask-Animate reaches SSIM 0.785 and FID-VID 19.51, versus 0.714 and 21.75 for MagicAnimate, a gain of 0.067 SSIM and 2.24 FID-VID over the best prior models.
- It achieves the best L1 (0.72E-04) and PSNR (18.61) on TikTok and ranks second on LPIPS (0.261).
- Ablations show removing the S-Adapter hurts most (SSIM 0.735 vs 0.785 full; FID-VID 25.10 vs 19.51).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ReMask-Animate instead of MagicPose | Human image animation · TikTok | SSIM: 0.752 -> 0.785 | Helped |
| Use ReMask-Animate instead of MagicAnimate | Human image animation · TikTok | LPIPS: 0.239 -> 0.261 | Hurt |
| Add P-Adapter for mask-guided pose feature fusion (ablation) | ReMask-Animate · TikTok | SSIM: 0.771 -> 0.785 | Helped |
| Add P-Adapter for mask-guided pose feature fusion (ablation) | ReMask-Animate · TikTok | FID-VID: 22.77 -> 19.51 | Helped |
| Add S-Adapter for foreground/background layout guidance (ablation) | ReMask-Animate · TikTok | SSIM: 0.735 -> 0.785 | Helped |
| Add S-Adapter for foreground/background layout guidance (ablation) | ReMask-Animate · TikTok | FID-VID: 25.10 -> 19.51 | Helped |
| Add C-Adapter for identity feature injection (ablation) | ReMask-Animate · TikTok | SSIM: 0.769 -> 0.785 | Helped |
| Add C-Adapter for identity feature injection (ablation) | ReMask-Animate · TikTok | FID-VID: 23.10 -> 19.51 | Helped |
| Add skeletal pose condition (ablation) | ReMask-Animate · TikTok | SSIM: 0.765 -> 0.785 | Helped |
| Add mesh pose condition (ablation) | ReMask-Animate · TikTok | SSIM: 0.777 -> 0.785 | Helped |
Limitations
- Mask-guided S-Adapter masks are not available at inference and are approximated by a self-evolution process using a segmentation network.
- Evaluation limited to TikTok and Fashion datasets; Fashion results are described as only comparable to other methods.
- Mask generation and pose handling depend on DWPose and SMPL estimators.
Experimental setup
Datasets: TikTok Fashion
Benchmarks: TikTok Fashion
Metrics: L1 PSNR SSIM LPIPS FID-VID
Baselines: FOMM MRAA TPSMM DreamPose DisCo MagicAnimate MagicPose AnimateAnyone
Builds on: AnimateAnyone ControlNet IP-Adapter AnimateDiff MagicAnimate
Closest papers by meaning
- Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation (CVPR 2024 · similarity 0.88)
- FaceShot: Bring Any Character into Life (ICLR 2025 · similarity 0.87)
- HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.86)
- DisPose: Disentangling Pose Guidance for Controllable Human Image Animation (ICLR 2025 · similarity 0.86)
- EverybodyDance: Bipartite Graph–Based Identity Correspondence for Multi-Character Animation (NeurIPS 2025 · similarity 0.86)
- Free-viewpoint Human Animation with Pose-correlated Reference Selection (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.