Research map / Video / Video generation
FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FaithfulFaces, a pose-shared identity aligner (dictionary + contrastive pose-invariance loss + Euler angle embeddings) injected into VACE-14B via LoRA to improve identity preservation under pose changes and occlusions in identity-preserving text-to-video generation, with a new pose-diverse video dataset of 51,624 samples.
Problem
Identity-preserving text-to-video generation often shows identity distortion and facial collapse under large facial pose variations and occlusions, since single-view reference encoders capture only one pose view.
Approach
A pose-shared identity aligner maps face embeddings (augmented with Euler angle embeddings) onto a learned shared dictionary, and is trained with a contrastive loss treating same-identity different-pose face crops as positives, yielding a global pose-faithful identity prior injected into a DiT video generator.
Key results
- FaithfulFaces reaches FaceSim-Cur 0.568 and FaceSim-Arc 0.542 on 600 generated videos, versus 0.484/0.456 for the best baseline Phantom-14B.
- FID drops to 164.24 from 191.02 (VACE-14B) and CLIPScore rises to 33.93, the best among compared methods.
- Ablation: removing Euler embeddings lowers FaceSim-Cur from 0.568 to 0.522; removing both aligner and Euler drops it to 0.437.
Surprising findings
- ArcFace features used in place of the learned aligner features give only modest gains over the baseline, and CLIP features perform worse than no aligner in FaceSim terms (0.447 vs 0.437 baseline is close).
Limitations
- Evaluation uses only 30 identities and 20 prompts (600 videos), a small test set.
- Dataset is curated from internet and in-house sources and is not stated as released.
- Robustness to Euler angle errors degrades beyond about 15 degrees perturbation.
Experimental setup
Datasets: in-house and internet face videos (curated, 51,624 samples) ConsisID evaluation face images
Benchmarks: FaceSim-Cur FaceSim-Arc FID CLIPScore
Metrics: FaceSim-Cur FaceSim-Arc FID CLIPScore
Baselines: Vidu Kling ConsisID VACE-14B HunyuanCustom Phantom-14B Concat-ID-Wan SkyReels-A2 Stand-In MAGREF
Builds on: VACE flow matching InfoNCE 6DRepNet ConsisID
Closest papers by meaning
- Identity-Preserving Text-to-Video Generation by Frequency Decomposition (CVPR 2025 · similarity 0.89)
- StableAnimator: High-Quality Identity-Preserving Human Image Animation (CVPR 2025 · similarity 0.88)
- ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation (NeurIPS 2026 · similarity 0.88)
- When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation (CVPR 2024 · similarity 0.87)
- PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization (CVPR 2024 · similarity 0.87)
- Stable-Pose: Leveraging Transformers for Pose-Guided Text-to-Image Generation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.