Research map / 3D vision / Face and shape
Pose Adapted Shape Learning for Large-Pose Face Reenactment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PASL, a face reenactment framework with six pose-adapted identity encoders, a cycle-consistent 3D shape generator (built on DECA), and an attention-embedded generator, plus new large-pose benchmarks MPIE-LP and VoxCeleb2-LP.
Problem
Face reenactment degrades under large head poses because standard face encoders are trained on pose-imbalanced data, so identity is poorly preserved for profile faces, and existing benchmarks contain too few large-pose pairs to evaluate this.
Approach
Replace a single identity encoder with six pose-pair-specific Pose-Adapted Encoders used in the identity loss, combine them with a cycle-consistent recomposition of identity and action shape codes (from DECA) and a transformer-based attention generator, and build large-pose evaluation subsets from MPIE and VoxCeleb2.
Key results
- On MPIE-LP cross-reenactment, PASL reaches FID 18.1 and CSIM 0.27/0.24/0.46 (ArcFace/VGGFace2/PAE), outperforming DG (FID 24.2) and other baselines.
- On VoxCeleb1 self-reenactment, PASL achieves FID 32.4 and CSIM 0.71 (ArcFace per the table), beating DG (FID 41.1) and HyperReenact (FID 38.2).
- The PAE encoder achieves higher verification accuracy than ArcFace, MagFace, VGGFace2, and AdaFace on all six pose test subsets, e.g., 73.8% vs 28.1% for ArcFace on profile-vs-profile pairs (Table 1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cycle-consistent shape loss Lcc (CSG) (ablation) | shape generator · MPIE-LP self-reenactment | FID: 23.9 -> 18.1 | Helped |
| add attention-embedded generator (AEG) instead of StarGAN2 (ablation) | generator · MPIE-LP self-reenactment | FID: 28.4 -> 18.1 | Helped |
| use PAE identity loss instead of off-the-shelf encoders (ablation) | cross-reenactment training · MPIE-LP cross-reenactment | CSIM (PAE): PAE 0.491 vs VGGFace2 0.466 (ss) | Helped |
| use PAE instead of ArcFace/MagFace/VGGFace2 for verification (ablation) | face encoder · six pose test subsets (IJB-C) | verification accuracy: PAE 92.2 FF, 73.8 PP vs VGGFace2 86.7, 60.7 | Helped |
| fine-tune off-the-shelf encoders on PAE pose-subset training data (ablation) | VGGFace2 encoder · FS pose subset verification | verification accuracy: VGGFace2 84.9 vs PAE 80.1 on FS | Mixed |
| PASL full method vs baselines | reenactment · MPIE-LP, VoxCeleb1, VoxCeleb2-LP self and cross reenactment | FID, CSIM, ARD, LPIPS: PASL best FID on all three benchmarks | Helped |
| use FOMM-style or landmark-based baselines instead of PASL | large-pose reenactment · MPIE-LP self-reenactment | FID: FOMM 62.7, Bi-layer 106.8 vs PASL 17.5 | Hurt |
Surprising findings
- Off-the-shelf ArcFace drops to 28.1% verification accuracy on profile-vs-profile pairs, while fine-tuning other encoders on pose-balanced data does not fully close the gap to PAE.
Limitations
- Large-pose benchmarks are constructed from MPIE (127 subjects) and a subset of VoxCeleb2, so evaluation scale for MPIE-LP is small.
- VoxCeleb1 has too few large-pose samples to assess large-pose performance.
- Several baselines were not re-run; numbers were copied from their papers, complicating direct comparison.
Experimental setup
Datasets: MS1M-V3 MPIE VoxCeleb1 VoxCeleb2 IJB-C
Benchmarks: MPIE-LP VoxCeleb1 VoxCeleb2-LP IJB-C
Metrics: FID CSIM ARD LPIPS verification accuracy
Baselines: FOMM Bi-layer DG HyperReenact HeadGAN Face2Face-rho ArcFace MagFace AdaFace
Builds on: DECA FLAME StarGAN v2 VGGFace2
Closest papers by meaning
- Dual-Generator Face Reenactment (CVPR 2022 · similarity 0.89)
- PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction (ICLR 2024 · similarity 0.83)
- FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation (NeurIPS 2026 · similarity 0.83)
- ControlFace: Harnessing Facial Parametric Control for Face Rigging (CVPR 2025 · similarity 0.82)
- Learning Dense Correspondence for NeRF-Based Face Reenactment (AAAI 2024 · similarity 0.82)
- FSRT: Facial Scene Representation Transformer for Face Reenactment from Factorized Appearance Head-pose and Facial Expression Features (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.