Research map / Detection and segmentation / Open-set and few-shot detection
Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds the FairFD racial-bias benchmark for face forgery detection (11 forgery methods, race-balanced RFW source faces), proposes Approach Averaged and Utility Regularized fairness metrics, and a training-free pruning method BPFA that improves fairness of detectors.
Problem
Racial bias in deep face forgery detectors is under-evaluated; existing datasets have few subjects, undefined ethnicity labels, and few forgery methods, and naive fairness metrics can be misleading (bias offset and aggregation distortion).
Approach
A race-balanced, multi-forgery benchmark (FairFD) plus fairness metrics computed per forgery approach and regularized by utility, and a post-hoc pruning score that favors weights with low magnitude and high cross-race activation variability, requiring no retraining.
Key results
- Across 12 detectors on FairFD, SPSL has the lowest naive DPD (0.0203) but the least-fair UCF reaches 0.1765 DPD; the URDPD of UCF is 4.76x that of SPSL.
- BPFA applied to SPSL reduces URDPD from 0.0265 baseline-to-0.0265 and raises AUC to 0.6862 and ACC to 0.8055 (vs original 0.6763/0.7618), achieving a new fairness SOTA.
- Pruning baseline RoBA degrades utility severely (e.g., ACC 0.3468 at 0.1% pruning), while BPFA keeps utility while improving fairness metrics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply BPFA pruning at 7% | SPSL on FairFD · FairFD | AADPD: 0.0556 -> 0.0473 | Helped |
| Apply BPFA pruning at 7% | SPSL on FairFD · FairFD | AUC: 0.6763 -> 0.6862 | Helped |
| Prune by reciprocal activation bias instead of BPFA | SPSL on FairFD · FairFD | AADPD: 0.0556 -> 0.1462 | Hurt |
| Train on race-balanced data instead of unbalanced data (ablation) | Xception on FairFD · FairFD | AADPD: 0.1957 -> 0.1030 | Helped |
| Train on race-balanced data instead of unbalanced data (ablation) | Xception on FairFD · FairFD | Accuracy: 0.5552 -> 0.3984 | Hurt |
| Train on race-balanced data instead of unbalanced data (ablation) | F3Net on FairFD · FairFD | AADPD: 0.1102 -> 0.0996 | Helped |
| Train on race-balanced data instead of unbalanced data (ablation) | F3Net on FairFD · FairFD | Accuracy: 0.5266 -> 0.3308 | Hurt |
| Use a race-specific optimal threshold instead of 0.5 (ablation) | Xception on FairFD real faces and FaceSwap · FairFD real faces and FaceSwap | AADPD: 0.1274 -> 0.0544 | Helped |
| Convert input images from RGB to grayscale (ablation) | Xception on FairFD · FairFD | DPD: 0.1538 -> 0.1309 | Helped |
| Convert input images from RGB to grayscale (ablation) | F3Net on FairFD · FairFD | DPD: 0.0764 -> 0.0780 | Hurt |
| Increase BPFA pruning rate from 1% to 4% (ablation) | SPSL on FairFD · FairFD | AADPD: 0.0556 -> 0.0523 | Helped |
| Increase BPFA pruning rate from 7% to 10% (ablation) | SPSL on FairFD · FairFD | AADPD: 0.0473 -> 0.0487 | Hurt |
Surprising findings
- Training on a race-balanced dataset does not reliably eliminate racial bias and reduces AUC, and for some metrics (F3Net) bias does not improve.
- Converting RGB to grayscale reduces bias for spatial detectors but not for frequency-based detectors, suggesting color information drives spatial-detector bias.
Limitations
- Image-level dataset only; video-level forgery not covered.
- Racial labels limited to four groups; attributes not perfectly balanced within groups.
- Balanced training data experiments are limited by small subject counts from FF++ for African and Indian groups.
Experimental setup
Datasets: FF++ (c23) RFW FairFD
Benchmarks: FairFD
Metrics: DPD DEOdds DEO STD Approach Averaged Metric (AADPD, AADEOdds, AADEO, AASTD) Utility Regularized Metric (URDPD, URDEOdds, URDEO, URSTD) AUC ACC
Baselines: WEIG RoBA DAG DAW PFGDFD Xception RECCE UCF Capsule FFD CORE F3Net
Builds on: Demographic Parity Equalized Odds RFW FaceForensics++ magnitude pruning
Closest papers by meaning
- Preserving Fairness Generalization in Deepfake Detection (CVPR 2024 · similarity 0.86)
- BIAS-ID: A Framework for Analyzing Transformation Biases in AI-Generated Image Detectors (NeurIPS 2026 · similarity 0.86)
- FairDD: Fair Dataset Distillation (NeurIPS 2025 · similarity 0.85)
- Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing (AAAI 2025 · similarity 0.85)
- Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models (CVPR 2025 · similarity 0.85)
- Leveraging Diffusion Perturbations for Measuring Fairness in Computer Vision (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.