Research map / Federated learning and privacy / Privacy attacks and secure learning

How Private is Private? A Comparative Study for Face De-Identification

NeurIPS 2026 · Federated learning and privacy › Privacy attacks and secure learning · approach: Visual privacy and face de-identification

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UtilFace, a demographically balanced face benchmark built from four large face-recognition datasets, and HiFD, a consistency-based hierarchical metric (privacy, quality, three utility levels) aggregated by weighted harmonic mean, used to compare 12 face de-identification methods.

Problem

Face de-identification evaluation is fragmented: methods are measured on different datasets, attributes and metrics, so cross-method comparison of privacy and multi-level utility preservation is infeasible.

Approach

Replace label-based utility accuracy with consistency between pretrained estimators' outputs on original and de-identified faces, organize utility into macro, micro and imperceptible (rPPG) levels, and aggregate five components with a weighted harmonic mean so that any near-zero component (e.g., identity suppression) vetoes the composite score.

Key results

• TIP-IM achieves the highest HiFD of 0.607, the only method reaching at least 0.44 on every dimension.
• Adv-Makeup has near-zero privacy (P=0.012) yet top arithmetic-mean score 0.673; harmonic mean ranks it 12th at 0.056.
• Consistency score predicts whether macro-expression label survives on de-identified RAF-DB with AUC 0.98 over 28,614 pairs.

Surprising findings

• Adversarial methods such as MI-FGSM and PGD stall at privacy below 0.14 against a modern recognizer ensemble, de-identifying only about 8% of faces.
• Macro-level utility (L1) is retained by most methods while rPPG (L3) is destroyed by nearly all generative methods, and the asymmetry holds across thresholds.

Limitations

• HiFD inherits biases of the pretrained estimators used for each consistency score.
• UtilFace carries no attribute annotations, and source datasets carry research-only licenses with consent concerns.
• Privacy is measured only against machine recognizers, not human observers or future recognizers.

Experimental setup

Datasets: WebFace42M Glint360K MS1MV2 MegaFace DFME PURE RAF-DB MMPD FFHQ

Benchmarks: UtilFace DFME PURE RAF-DB MMPD

Metrics: HiFD privacy score (ensemble cosine dissimilarity) quality (LPIPS and NIQE) age/gender/ethnicity/expression/landmark consistency gaze and micro-expression consistency rPPG BVP and HR consistency de-identification success rate at FAR 1e-3

Baselines: MI-FGSM PGD TI-DIM TIP-IM Chameleon CIAGAN AMT-GAN Adv-Makeup DeID-rPPG G2Face DiffAM WeakenDiff

Builds on: LPIPS NIQE ArcFace PhysNet GFPGAN

Closest papers by meaning

• Disguise without Disruption: Utility-Preserving Face De-identification (AAAI 2024 · similarity 0.90)
• Privacy-Preserving Optics for Enhancing Protection in Face De-Identification (CVPR 2024 · similarity 0.87)
• Validating Privacy-Preserving Face Recognition under a Minimum Assumption (CVPR 2024 · similarity 0.86)
• $\text{ID}^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition (NeurIPS 2024 · similarity 0.86)
• DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection (CVPR 2024 · similarity 0.86)
• Privacy-Preserving Face Recognition Using Trainable Feature Subtraction (CVPR 2024 · similarity 0.85)