Research map / Video / Video generation
ShowMaker: Creating High-Fidelity 2D Human Video via Fine-Grained Diffusion Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ShowMaker, a dual-stream diffusion framework that generates half-body conversational human videos from 2D pose keypoints, adding a key point-based hand modeling module and a Face Recapture module for hands and face fidelity.
Problem
Existing human video generation methods handle either face or full body and produce low-fidelity hands and faces, and suffer identity leakage in cross-identity animation, which limits realistic conversational human videos with gestures and facial movements.
Approach
Uses 2D keypoint-driven dual-stream diffusion with Fourier-encoded hand keypoint coordinates and a keypoint-indexed orthogonal learnable codebook injected via hand cross-attention, plus a Face Recapture module fusing texture (VAE) and ArcFace identity features into face attention.
Key results
- Achieves FVD 197.43 vs 365.83 for AnimateAnyone and 370.75 for MagicPose; FID 15.43 and SSIM 0.85 on the main test set.
- On Seth, ShowMaker reaches FID 9.83 and FVD 193.25 versus 32.32 and 428.84 for Make-Your-Anchor.
- Ablation: removing the hand module raises hand keypoint error from 0.77 to 1.12; removing stage 2 raises FVD from 197.43 to 373.87.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ShowMaker instead of MagicPose | 2D human video generation · Talkshow and indoor test clips | SSIM: 0.82 -> 0.85 | Helped |
| use ShowMaker instead of MagicPose | 2D human video generation · Talkshow and indoor test clips | PSNR: 30.03 -> 32.23 | Helped |
| use ShowMaker instead of AnimateAnyone | 2D human video generation · Talkshow and indoor test clips | FVD: 365.83 -> 197.43 | Helped |
| use ShowMaker instead of AnimateAnyone | 2D human video generation · Talkshow and indoor test clips | Lhand: 1.10 -> 0.77 | Helped |
| add Key Point-based Fine-grained Hand Modeling (ablation) | ShowMaker · Seth test clips | Lhand: 1.12 -> 0.77 | Helped |
| add Key Point-based Fine-grained Hand Modeling (ablation) | ShowMaker · Seth test clips | FVD: 212.25 -> 197.43 | Helped |
| add Face Recapture (ablation) | ShowMaker · Seth test clips | Lface: 0.23 -> 0.19 | Helped |
| add Face Recapture (ablation) | ShowMaker · Seth test clips | FVD: 208.28 -> 197.43 | Helped |
| use two-stage training, including Stage 2 (ablation) | ShowMaker · Seth test clips | FVD: 373.87 -> 197.43 | Helped |
| use cropped hand images as hand-texture compensation instead of the proposed hand modeling (ablation) | ShowMaker · Hand synthesis | hand texture and structure quality | No clear effect |
| remove positional encoding from the hand modeling module (ablation) | ShowMaker · Hand synthesis | hand detail and structural accuracy | Hurt |
Limitations
- Performance degrades when DWPose keypoints are corrupted by severe motion blur, causing artifacts.
- Less satisfactory results on challenging cases such as reflections on glasses.
- Only evaluated on two TalkShow actors and seven indoor-recorded subjects; no error bars reported.
Experimental setup
Datasets: TalkShow (Seth, Oliver) indoor recordings of seven people
Benchmarks: TalkShow (Seth, Oliver) test split
Metrics: SSIM PSNR FID FVD body/face/hand keypoint Euclidean error user preference score
Baselines: TPS Disco AnimateAnyone MagicAnimate MagicPose Make-Your-Anchor
Builds on: Stable Diffusion AnimateDiff DWPose ArcFace
Closest papers by meaning
- VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis (CVPR 2025 · similarity 0.89)
- Make-Your-Anchor: A Diffusion-based 2D Avatar Generation Framework (CVPR 2024 · similarity 0.88)
- SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios (NeurIPS 2025 · similarity 0.88)
- Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model (CVPR 2025 · similarity 0.88)
- RealPortrait: Realistic Portrait Animation with Diffusion Transformers (AAAI 2025 · similarity 0.87)
- AudCast: Audio-Driven Human Video Generation by Cascaded Diffusion Transformers (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.