Research map / Image generation / Editing and style transfer
Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Vec2Face trains a feature masked autoencoder with image decoder and GAN loss to generate face images from identity vectors, sampling well-separated vectors and perturbing them to create the HSFace synthetic face recognition datasets (up to 300K identities).
Problem
Synthesizing face images of non-existent persons for training face recognition models, requiring many well-separated identities (inter-class separation) and proper appearance variation per identity (intra-class variation), which prior methods struggle to provide at scale.
Approach
A holistic model that maps a 512-d face feature vector (from a pretrained FR model) to an image via a feature masked autoencoder and image decoder; identities are sampled as vectors with cosine similarity below 0.3, and small Gaussian perturbations of identity vectors yield same-identity variation, with AttrOP gradient-based vector adjustment for pose and quality control.
Key results
- HSFace10K (0.5M images, 10K IDs) achieves 92.00% average accuracy over five real-world test sets, state of the art among compared synthetic datasets, and 83.82% / 86.96% on IJBB / IJBC.
- Scaling to HSFace300K (15M images, 300K IDs) raises average accuracy to 93.52%, with gains until 300K identities.
- Vec2Face runs at 467 FPS with a 0.68GB model versus 1.5 FPS and 3.4GB for Arc2Face (311x faster).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale HSFace from 10K to 300K identities | Vec2Face-generated HSFace training data · Average of five test sets | accuracy: 92.00 -> 92.47 -> 92.97 -> 93.22 -> 93.52 | Helped |
| Scale HSFace from 300K to 400K identities | Vec2Face-generated HSFace training data · Average of five test sets | accuracy: 93.52 -> 93.24 | Hurt |
| Add AttrOP quality control at Q=27 (ablation) | Vec2Face-generated training set · Average of five test sets | accuracy: 88.04 -> 89.98 | Helped |
| Add AttrOP pose control at 60° (ablation) | Vec2Face-generated training set with quality control · Average of five test sets | accuracy: 89.98 -> 91.50 | Helped |
| Add AttrOP pose control at 85° (ablation) | Vec2Face-generated training set with 60° pose control · Average of five test sets | accuracy: 91.50 -> 92.00 | Helped |
| Add identity loss L_id (ablation) | Vec2Face · Face reconstruction, especially large-pose images | identity preservation | Helped |
| Add perceptual loss L_lpips (ablation) | Vec2Face · Face reconstruction | face structure clarity | Helped |
| Use GAN loss only after 1,000 epochs instead of early in training (ablation) | Vec2Face · Face reconstruction | training convergence | Helped |
| Increase perturbation variance σ (ablation) | Vec2Face-generated HSFace identities · Face recognition accuracy | accuracy | Mixed |
| Increase the sampling percentage of large-σ perturbations (ablation) | Vec2Face-generated HSFace identities · Face recognition accuracy | accuracy | Hurt |
| Replace 1,194 identities similar to Vec2Face training identities (ablation) | HSFace300K · Face recognition performance | accuracy | No clear effect |
Surprising findings
- Models trained on HSFace10K outperform those trained on same-scale real CASIA-WebFace on CALFW, IJBB and IJBC.
- Too large identity separation (beyond a threshold) does not further improve FR performance.
Limitations
- Less useful for fine-grained tasks such as discriminating doppelgangers (SLLFW, DoppelVer).
- Identity definition via similarity threshold is heuristic; threshold depends on FR model.
- Synthetic data still underperforms real data on some benchmarks (e.g., CFP-FP) and combined use is needed for best results.
Experimental setup
Datasets: WebFace4M MS1MV2 LFW CFP-FP CPLFW AgeDB-30 CALFW Hadrian Eclipse SLLFW DoppelVer IJBB
Benchmarks: LFW CFP-FP CPLFW AgeDB-30 CALFW IJBB IJBC Hadrian Eclipse SLLFW DoppelVer
Metrics: verification accuracy TPR@FPR=1e-4 average accuracy FID FPS
Baselines: IDiff-Face DCFace ID^3 Arc2Face DigiFace SynFace SFace IDnet ExFaceGAN SFace2 CASIA-WebFace
Builds on: MAE ArcFace Arc2Face DCFace MagFace
Closest papers by meaning
- HyperFace: Generating Synthetic Face Recognition Datasets by Exploring Face Embedding Hypersphere (ICLR 2025 · similarity 0.90)
- UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition (ICLR 2025 · similarity 0.89)
- CemiFace: Center-based Semi-hard Synthetic Face Generation for Face Recognition (NeurIPS 2024 · similarity 0.88)
- $\text{ID}^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition (NeurIPS 2024 · similarity 0.87)
- GIF: Generative Inspiration for Face Recognition at Scale (CVPR 2025 · similarity 0.87)
- FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.