Research map / Label-efficient and robust learning / Recognition and long-tail
Masked Face Recognition with Generative-to-Discriminative Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes G2D, a three-module network (ICT-based generative encoder, CNN discriminative reformer distilled from VGGFace2, and a classifier head) pretrained greedily module-wise on synthetic masked faces for masked face recognition.
Problem
Masked face recognition suffers from insufficient and inaccurate identity representations caused by diverse occlusions, and existing generative inpainting and discriminative finetuning approaches fail to give both general and identity-discriminative features.
Approach
Cascade a generative encoder (finetuned ICT inpainting encoder trained with reconstruction and adversarial losses) with a discriminative reformer trained by relational knowledge distillation from a pretrained face recognizer, then finetune a classifier head, all learned greedily module-wise on synthetic masked faces.
Key results
- On synthetic masked LFW under UMR-MP, G2D reaches 97.75% accuracy and 3.05% EER, versus 96.01% for the best listed baseline (ICT+ArcFace under MR-MP).
- On RMFD realistic masked faces (UMR-MP), G2D achieves 79.18% accuracy versus 72.55% for DoDGAN and 72.22% for VGGFace2.
- G2D reaches 92.23% on CPLFW, competitive with normal face recognition baselines (MaskInv 92.86%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use G2D instead of ICT+ArcFace | masked face verification · Synthetic masked LFW MR-MP | ACC: 96.01% -> 97.58% | Helped |
| use G2D instead of ICT+VGGFace2 | masked face verification · Synthetic masked LFW UMR-MP | ACC: 94.99% -> 97.75% | Helped |
| use G2D instead of DoDGAN | masked face verification · Realistic masked RMFD UMR-MP | ACC: 72.55% -> 79.18% | Helped |
| replace the ICT generative encoder with a DeepFill CNN encoder (ablation) | G2D · Synthetic masked LFW UMR-MP | ACC: 97.75% -> 96.42% | Hurt |
| remove convolutional layers from the ICT upsampling network (ablation) | G2D · Synthetic masked LFW UMR-MP | ACC: 97.75% -> 93.99% | Hurt |
| train with classification loss only (ablation) | G2D · Synthetic masked LFW UMR-MP | ACC: 97.75% -> 83.50% | Hurt |
| train the reformer with L1 feature loss only (ablation) | G2D · Synthetic masked LFW UMR-MP | ACC: 97.75% -> 95.25% | Hurt |
| replace the ICT generative encoder with a DeepFill CNN encoder (ablation) | G2D · Synthetic masked LFW MR-MP | ACC: 97.58% -> 96.14% | Hurt |
| remove convolutional layers from the ICT upsampling network (ablation) | G2D · Synthetic masked LFW MR-MP | ACC: 97.58% -> 91.48% | Hurt |
| train with classification loss only (ablation) | G2D · Synthetic masked LFW MR-MP | ACC: 97.58% -> 82.72% | Hurt |
| train the reformer with L1 feature loss only (ablation) | G2D · Synthetic masked LFW MR-MP | ACC: 97.58% -> 93.53% | Hurt |
| replace the ICT generative encoder with a DeepFill CNN encoder (ablation) | G2D · Realistic masked RMFD UMR-MP | ACC: 79.18% -> 73.26% | Hurt |
Surprising findings
- Directly enforcing the reformer to approximate identity labels (G2D[CE]) yields much lower accuracy (83.50%) than distillation from a teacher.
- Finetuned masked-face recognizers (SRT, DoDGAN) show lower cross-quality performance than their original unmodified baselines.
Limitations
- Model has 178.5M parameters, about 7x the 25.6M of ResNet50-based general recognizers.
- Relies on synthetic masks from a collection of 45 transparent mask images, which may not cover real mask diversity.
Experimental setup
Datasets: Celeb-A FFHQ MaskTheFace
Benchmarks: LFW (synthetic masked) RMFD MLFW CPLFW
Metrics: ACC EER FDR FMR100 FMR1000 FNMR AVG
Baselines: CenterLoss VGGFace VGGFace2 ArcFace DeepFill ICT DoDGAN Self-Restrained Loss (SRT) IDGAN MaskInv
Builds on: ICT MaskTheFace WGAN-GP
Closest papers by meaning
- FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learning (CVPR 2025 · similarity 0.89)
- Detecting Masked Faces in the Wild With LLE-CNNs (CVPR 2017 · similarity 0.88)
- UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition (ICLR 2025 · similarity 0.87)
- Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors (ICLR 2025 · similarity 0.87)
- HyperFace: Generating Synthetic Face Recognition Datasets by Exploring Face Embedding Hypersphere (ICLR 2025 · similarity 0.86)
- GIF: Generative Inspiration for Face Recognition at Scale (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.