Research map / 3D vision / Face and shape
EFHQ: Multi-purpose ExtremePose-Face-HQ dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces EFHQ, a ~450k-frame extreme-pose high-quality face dataset curated from VFHQ and CelebV-HQ via a multi-estimator pose binning pipeline, and validates it on face generation, reenactment, and a cross-pose verification benchmark.
Problem
Existing large face datasets are dominated by near-frontal poses, so generators, reenactment models, and face recognition systems degrade on profile or pitched faces.
Approach
Curate extreme-pose frames from existing high-resolution face videos using an ensemble of pose estimators (SynergyNet, DirectMHP, in-house FacePoseNet) with majority-vote binning, human review, and identity labels, producing a multi-purpose dataset that supports generation, reenactment, and verification.
Key results
- Face recognition on the EFHQ verification benchmark: TAR@FAR=1e-3 drops by 5-37% from frontal-frontal to frontal-profile subsets across ArcFace/AdaFace models.
- StyleGAN2-ADA trained on FFHQ+EFHQ conditionally achieves FID 3.33 on the combined set vs 3.44/8.26 for alternatives in unconditional comparisons, with improved Recall over LPFF-trained models.
- User study: EFHQ-trained reenactment models rated best or tied in identity (74.80%) and motion (64.22%) criteria.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add EFHQ to VoxCeleb1 training data | TPS · EFHQ | AKD: 3.814 -> 1.963 | Helped |
| Add EFHQ to VoxCeleb1 training data | TPS · VoxCeleb1 | L1: 0.041 -> 0.041 | No clear effect |
| Add EFHQ to VoxCeleb1 training data | LIA · EFHQ | AKD: 3.807 -> 2.291 | Helped |
| Add EFHQ to VoxCeleb1 training data | LIA · VoxCeleb1 | L1: 0.044 -> 0.044 | No clear effect |
| Fine-tune ControlNet on EFHQ | Stable Diffusion v1.5 ControlNet · EFHQ | FID: 111.23 -> 81.90 | Helped |
| Fine-tune ControlNet on EFHQ | Stable Diffusion v1.5 ControlNet · LPFF | FID: 119.74 -> 83.39 | Helped |
| Add EFHQ to FFHQ training data | StyleGAN2-ADA · FFHQ+EFHQ | Recall: 0.44 -> 0.46 | Helped |
| Add EFHQ to FFHQ training data | EG3D · FFHQ | Depth: 0.288 -> 0.224 | Helped |
| Add EFHQ to FFHQ training data | EG3D · FFHQ | FID: 3.7 -> 3.9 | Mixed |
Surprising findings
- The ResNet100 model trained on Glint360K performed worse than smaller models on the profile-profile subset, suggesting profile data is needed to avoid overfitting.
Limitations
- Final dataset released only as metadata files due to licensing of VFHQ and CelebV-HQ, requiring users to re-download source videos.
- Pose estimators are unstable at extreme angles, requiring ensemble and manual review.
- Identity consistency drops when reference data contains profile images because the face recognition model is weak on profiles.
Experimental setup
Datasets: VFHQ CelebV-HQ FFHQ LPFF VoxCeleb1 CPLFW CFP-FP
Benchmarks: EFHQ verification benchmark CPLFW FID Recall VoxCeleb1 reenactment test EFHQ reenactment test
Metrics: FID Recall NME MSE ID cosine similarity L1 AKD AED TAR@FAR=1e-3
Baselines: LPFF FFHQ-trained models TPS LIA ArcFace AdaFace
Builds on: StyleGAN2-ADA EG3D ControlNet VFHQ CelebV-HQ
Closest papers by meaning
- Pseudo Facial Generation With Extreme Poses for Face Recognition (CVPR 2021 · similarity 0.84)
- FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction (CVPR 2023 · similarity 0.83)
- UV-GAN: Adversarial Facial UV Map Completion for Pose-Invariant Face Recognition (CVPR 2018 · similarity 0.83)
- FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs (CVPR 2025 · similarity 0.82)
- HyperFace: Generating Synthetic Face Recognition Datasets by Exploring Face Embedding Hypersphere (ICLR 2025 · similarity 0.81)
- Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors (ICLR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.