Research map / Label-efficient and robust learning / Self-supervised and contrastive
Self-Supervised Facial Representation Learning with Facial Region Awareness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Facial Region Awareness (FRA), a BYOL-based self-supervised facial pre-training framework that learns consistent global and local (facial-region) representations via learned heatmaps from facial mask embeddings, using a ResNet-50 backbone.
Problem
Learning general self-supervised facial representations for diverse facial analysis tasks; prior methods enforce only image-level consistency and overlook consistency of local facial regions.
Approach
Generate facial-region heatmaps by cosine similarity between per-pixel projected features and facial mask embeddings produced by a Transformer decoder from learnable positional queries; train these heatmaps via a deep-clustering-style pixel-to-cluster assignment loss between online and momentum networks, and enforce consistency of global and region-pooled embeddings across views.
Key results
- On AffectNet, FRA (LP) with ResNet-50 reaches 66.16% top-1 under fine-tuning, outperforming FaRL (ViT-B/16, 64.85%) and BYOL-style baselines; FRA (EAC) reaches 90.62% on FERPlus.
- On CelebA attribute recognition, FRA (FT) achieves 92.02% averaged accuracy, above BYOL (91.56%) and PCL (91.48%).
- On 300W face alignment, FRA achieves 2.91 full-set NME, comparable to STAR (2.87) with an hourglass backbone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add local consistency and semantic relation to global consistency (ablation) | BYOL · RAF-DB | accuracy: 87.82 -> 88.72 | Helped |
| add local consistency and semantic relation to global consistency (ablation) | BYOL · CelebA | accuracy: 90.63 -> 91.18 | Helped |
| add local consistency and semantic relation to global consistency (ablation) | BYOL · 300W | NME: 3.56 -> 3.14 | Helped |
| use only semantic relation instead of global consistency (ablation) | FRA · RAF-DB | accuracy: 87.82 -> 85.34 | Hurt |
| use only semantic relation instead of global consistency (ablation) | FRA · 300W | NME: 3.56 -> 3.25 | Helped |
| use only local consistency instead of global consistency (ablation) | FRA · RAF-DB | accuracy: 87.82 -> 87.46 | No clear effect |
| use only local consistency instead of global consistency (ablation) | FRA · 300W | NME: 3.56 -> 3.19 | Helped |
| increase heatmap count from 8 to 32 or 64 (ablation) | FRA · RAF-DB | accuracy: 88.72 -> 88.36; 88.72 -> 88.18 | Hurt |
| increase heatmap count from 8 to 32 or 64 (ablation) | FRA · CelebA | accuracy: 91.18 -> 91.01; 91.18 -> 90.98 | Hurt |
| set λr to 0.5 or 1.0 instead of 0.1 (ablation) | FRA · RAF-DB | accuracy: 88.72 -> 88.45; 88.72 -> 88.08 | Hurt |
| set λr to 0.5 or 1.0 instead of 0.1 (ablation) | FRA · CelebA | accuracy: 91.18 -> 91.04; 91.18 -> 90.46 | Hurt |
| increase Transformer decoder layers from 1 to 2 or 3 (ablation) | FRA · RAF-DB | accuracy: 88.72 -> 89.01; 88.72 -> 89.06 | Helped |
Surprising findings
- Global-only consistency (BYOL) collapses classification performance on RAF-DB when local/region terms are used alone, showing global consistency is needed to avoid degeneration.
Limitations
- Heatmaps are only qualitatively shown to roughly discover facial regions, without quantitative part-localization evaluation.
- Ablations are performed only on RAF-DB and CelebA with VGGFace2 pre-training.
- Face alignment results are comparable to, not better than, SOTA hourglass-based methods.
Experimental setup
Datasets: VGGFace2 FERPlus RAF-DB AffectNet CelebA WFLW 300W
Benchmarks: FERPlus RAF-DB AffectNet CelebA 300W WFLW
Metrics: top-1 accuracy averaged attribute accuracy NME failure rate AUC
Baselines: BYOL LEWEL PCL Bulat et al. MCF FaRL EAC SLPT ADNet STAR
Builds on: BYOL MaskFormer SwAV LEWEL
Closest papers by meaning
- PrefAce: Face-Centric Pretraining with Self-Structure Aware Distillation (AAAI 2024 · similarity 0.88)
- FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learning (CVPR 2025 · similarity 0.88)
- LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition (CVPR 2024 · similarity 0.87)
- General Facial Representation Learning in a Visual-Linguistic Manner (CVPR 2022 · similarity 0.87)
- SegFace: Face Segmentation of Long-Tail Classes (AAAI 2025 · similarity 0.84)
- Masked Face Recognition with Generative-to-Discriminative Representations (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.