Research map / Image generation / Autoregressive and latent generation
AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces AI-Face, a million-scale AI-generated face dataset with skin tone, gender, and age annotations spanning deepfake videos, GANs, and diffusion models, and benchmarks fairness and reliability of 12 AI-face detectors.
Problem
Existing AI-generated face datasets lack demographic annotations together with diverse generation methods (deepfake videos, GANs, DMs), which hinders developing and evaluating fair AI-face detectors.
Approach
Builds a demographically annotated dataset by combining a landmark-and-color-based Monk Skin Tone estimator with a CLIP ViT-L/14 annotator for gender and age trained on IMDB-WIKI using vector-scaling imbalance loss, a Sinkhorn fairness loss, and SAM; then benchmarks 12 detectors on fairness, post-processing robustness, and cross-dataset generalization.
Key results
- PG-FDD achieves the best overall balance with AUC 99.172% and lowest FEO-style intersectional fairness among detectors in Table 4 (e.g., skin tone FEO 9.115%).
- Annotation quality of AI-Face gender ACC is 86.857% on strategic subset vs 8.143% for A-FF++, and age ACC 85.700% vs 37.700%.
- Detectors show severe degradation under post-processing and cross-domain evaluation; most detectors have the largest bias on diffusion-model-generated faces and higher false positive rates for lighter skin tones and female subgroups.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PG-FDD rather than the other evaluated detectors | AI-generated face detection · AI-Face test set | AUC | Helped |
| Use PG-FDD rather than the other evaluated detectors | AI-generated face detection · AI-Face test set, intersectional groups | FEO | Helped |
| Increase the training set size (ablation) | AI-Face detectors · AI-Face test set | AUC | Helped |
| Increase the training set size (ablation) | F3Net and UCF · AI-Face test set | FEO | Hurt |
| Increase the proportion of real samples (real-to-fake ratio 1:10 to 10:1) (ablation) | Most AI-Face detectors · AI-Face test set | FEO | Helped |
| Balance real and fake samples (ablation) | Most AI-Face detectors · AI-Face test set | AUC | Helped |
| Apply image post-processing (ablation) | AI-Face detectors · AI-Face test set | AUC | Hurt |
| Apply image post-processing (ablation) | UCF, UnivFD, CORE, and DAW-FDD · AI-Face test set | FEO | Mixed |
| Evaluate on external datasets instead of AI-Face (ablation) | AI-Face detectors · DF-Platter and GenData | AUC | Hurt |
| Evaluate on external dataset domains (ablation) | AI-Face detectors · CCv2, DF-Platter, and GenData | Fairness | Mixed |
Surprising findings
- Fairness-enhanced detectors maintain small overall fairness metrics yet still show biased detection toward minority groups.
- Increasing training set size improves utility but does not reliably improve fairness; some detectors get less fair with more data.
Limitations
- Demographic annotations are algorithmically generated and may not be 100% accurate.
- Dataset access is restricted to users who sign an end-user license agreement.
- Fairness-enhanced detectors degrade under post-processing and cross-domain evaluation; no reliable detector identified.
Experimental setup
Datasets: AI-Face IMDB-WIKI FFHQ FF++ DFDC DFD Celeb-DF-v2 CCv2 DF-Platter GenData
Benchmarks: AI-Face CCv2 DF-Platter GenData
Metrics: FDP FMEO FEO FOAE FIND AUC ACC AP EER FPR
Baselines: Xception EfficientB4 ViT-B/16 F3Net SPSL SRM UCF UnivFD CORE DAW-FDD DAG-FDD PG-FDD
Builds on: CLIP SAM (sharpness-aware minimization) Vector Scaling loss UnivFD PG-FDD
Closest papers by meaning
- MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humans (NeurIPS 2025 · similarity 0.87)
- HyperFace: Generating Synthetic Face Recognition Datasets by Exploring Face Embedding Hypersphere (ICLR 2025 · similarity 0.86)
- Limitations of Face Image Generation (AAAI 2024 · similarity 0.85)
- Fair Text-to-Image Diffusion via Fair Mapping (AAAI 2025 · similarity 0.85)
- AIM-Fair: Advancing Algorithmic Fairness via Selectively Fine-Tuning Biased Models with Contextual Synthetic Data (CVPR 2025 · similarity 0.85)
- Leveraging Diffusion Perturbations for Measuring Fairness in Computer Vision (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.