Research map / Video / Video generation
Enhancing Identity-Deformation Disentanglement in StyleGAN for One-Shot Face Video Re-Enactment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage StyleGAN2-based one-shot face video re-enactment model with conditional W+/SS latent disentanglement and a feature-space refinement network trained with latent-edit data augmentation.
Problem
One-shot face video re-enactment needs precise disentanglement of identity from facial deformation in latent space while preserving high-frequency facial details such as hair and wrinkles.
Approach
Encodes identity in W+ and deformation in SS space, uses a transformer-based conditional transformation network to produce source-conditioned deformation codes, and refines StyleGAN intermediate features with a gated, reference-aligned refinement network trained on latent-edited (hair/wrinkle) augmentations.
Key results
- Same-identity re-enactment on the Celeb-trained setting reaches PSNR 17.43 and CSIM 0.57 versus PASL's 12.01 and 0.33.
- The refinement network raises CSIM from 0.66 to 0.72 (about 9%) with AED changing from 9.8 to 9.7 on HDTF ablation.
- The transformation network lowers AED from 10.6 to 9.8 and APD from 0.19 to 0.17 relative to no transformation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add identity-based transformation network T for conditional deformation-code disentanglement (ablation) | StyleGAN2-based re-enactment model · HDTF, same-identity re-enactment | AED: 10.6 -> 9.8 | Helped |
| Add identity-based transformation network T for conditional deformation-code disentanglement (ablation) | StyleGAN2-based re-enactment model · HDTF, same-identity re-enactment | APD: 0.19 -> 0.17 | Helped |
| Add identity-based transformation network T for conditional deformation-code disentanglement (ablation) | StyleGAN2-based re-enactment model · HDTF, same-identity re-enactment | CSIM: 0.65 -> 0.66 (+1%) | No clear effect |
| Add refinement network R to adjust StyleGAN intermediate features (ablation) | StyleGAN2-based re-enactment model · HDTF, same-identity re-enactment | CSIM: 0.66 -> 0.72(+6%) | Helped |
| Add refinement network R to adjust StyleGAN intermediate features (ablation) | StyleGAN2-based re-enactment model · HDTF, same-identity re-enactment | AED: 9.8 -> 9.7(+0.1) | No clear effect |
| Use the full method rather than the strongest reported baseline | StyleGAN2-based re-enactment model · HDTF, cross-identity re-enactment | AED: 10.7 -> 9.1 | Helped |
| Use the full method rather than the strongest reported baseline | StyleGAN2-based re-enactment model · HDTF, cross-identity re-enactment | CSIM: 0.89 -> 0.79 | Mixed |
Surprising findings
- Compared with VOODOO3D and Potrait4Dv2 (trained on larger data), the method does not win on every metric; Potrait4Dv2 has better cross-identity FID (92.7 vs 118.6).
Limitations
- Requires face alignment because it is built on StyleGAN2.
- Modeling backgrounds in the StyleGAN2 latent space trained on faces remains a challenge.
- Potential misuse for deepfake generation from a single image.
Experimental setup
Datasets: CelebV-HQ HDTF Celeb (VoxCeleb-style, ~100k videos)
Benchmarks: HDTF CelebV-HQ Celeb
Metrics: L1 LPIPS PSNR SSIM CSIM FID AED APD
Baselines: PASL HyperReenact VOODOO3D Potrait4Dv2 LIA PIRender StyleHeat Robust
Builds on: StyleGAN2 e4e Robust (Oorloff and Yacoob 2023) HFGI
Closest papers by meaning
- When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation (CVPR 2024 · similarity 0.87)
- FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation (NeurIPS 2026 · similarity 0.86)
- SDGAN: Disentangling Semantic Manipulation for Facial Attribute Editing (AAAI 2024 · similarity 0.86)
- Synergizing Motion and Appearance: Multi-Scale Compensatory Codebooks for Talking Head Video Generation (CVPR 2025 · similarity 0.85)
- X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention (ICLR 2025 · similarity 0.85)
- Low-Rank Head Avatar Personalization with Registers (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.