Research map / Label-efficient and robust learning / Self-supervised and contrastive
LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAFS, a landmark-based self-supervised pretraining (DINO-style full-to-subset landmark consistency with landmark shuffle and coordinate perturbation augmentations) on 1M unlabelled WebFace faces, followed by CosFace finetuning for face recognition.
Problem
Face recognition models need good initial facial representations, especially when labels are scarce (few-shot, 1-shot) or when self-supervised learning with ViTs fails to scale to large identity-rich datasets.
Approach
Replaces random grid crops in DINO with landmark-localized part patches from a frozen Part fViT landmark CNN, learning consistency between the full landmark set (teacher) and a random subset (student), plus two landmark-specific augmentations (shuffle and coordinate perturbation) and a landmark-position regularizer during finetuning.
Key results
- On Webface4M, LAFS-pretrained Part fViT-B reaches 97.55 on IJB-C and 96.28 on AgeDB, beating AdaFace (97.39 IJB-C) and surpassing it by 0.16 on IJB-C and 0.25 on IJB-B.
- In 1% label 1-shot setting, LAFS Part fViT achieves 33.97 IJB-B TAR vs 18.51 for DINO Part fViT and 37.49 IJB-C vs 21.40 for Part fViT with DINO.
- On MS1MV3, LAFS Part fViT achieves 97.48 IJB-C, outperforming AdaFace by 0.39 IJB-C.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune with CosFace instead of no fine-tuning (ablation) | LAFS · LFW | accuracy: 75.20 -> 88.58 | Helped |
| Fine-tune with CosFace instead of no fine-tuning (ablation) | LAFS · CFP-FP | accuracy: 70.82 -> 76.04 | Helped |
| Fine-tune with DINO loss instead of no fine-tuning (ablation) | LAFS · LFW | accuracy: 75.20 -> 76.96 | Helped |
| Add landmark shuffle (ablation) | LAFS with Part fViT · IJB-B, 1% labelled data | TAR@FAR=1e-4: 38.05 -> 40.01 | Helped |
| Add landmark coordinate perturbation (α=2) (ablation) | LAFS with Part fViT · IJB-B, 1% labelled data | TAR@FAR=1e-4: 38.05 -> 39.44 | Helped |
| Increase landmark coordinate perturbation from α=2 to α=5 (ablation) | LAFS with Part fViT · IJB-B, 1% labelled data | TAR@FAR=1e-4: 39.44 -> 39.16 | Hurt |
| Use landmark shuffle (ablation) | DINO with standard fViT · IJB-B, 1% labelled data | TAR@FAR=1e-4: 36.27 -> 30.96 | Hurt |
| Add coordinate perturbation (α=2) (ablation) | DINO with standard fViT · IJB-B, 1% labelled data | TAR@FAR=1e-4: 36.27 -> 36.30 | No clear effect |
| Use DINO rather than supervised training | ResNet · MS1MV3, IJB-C | TAR@FAR=1e-4: 97.22 -> 97.01 | Hurt |
| Use LAFS pretraining | Part fViT · MS1MV3, IJB-C | TAR@FAR=1e-4: 97.29 -> 97.48 | Helped |
| Use LAFS pretraining | Part fViT · MS1MV3, MegaFace/id | accuracy: 98.96 -> 98.66 | Hurt |
Surprising findings
- Landmark shuffle improves Part fViT recognition performance while degrading standard fViT with DINO, contrary to prior findings that shuffling hurts ViTs.
- Self-supervised pretraining gives little or negative benefit for ResNet-100 on large-scale data (e.g., 97.22 supervised vs 97.01 with SSL on MS1MV3 full data).
Limitations
- Landmarks are fixed from a pretrained landmark CNN and are not refined during self-supervised training.
- Self-supervised gains are marginal or negative for ResNet-100 on large-scale data.
- Evaluated only on face recognition; transfer to other facial tasks left to future work.
Experimental setup
Datasets: WebFace42M WebFace4M MS1MV3 MS1MV2
Benchmarks: LFW CFP-FP AgeDB IJB-B IJB-C MegaFace
Metrics: TAR@FAR=1e-4 verification accuracy identification accuracy
Baselines: DINO CosFace AdaFace QAFace ArcFace PFC-0.3 Part fViT fViT ResNet-100 VPL MagFace FAN-Face
Builds on: DINO Part fViT CosFace AdaFace
Closest papers by meaning
- PrefAce: Face-Centric Pretraining with Self-Structure Aware Distillation (AAAI 2024 · similarity 0.88)
- Self-Supervised Facial Representation Learning with Facial Region Awareness (CVPR 2024 · similarity 0.87)
- Brute-Force Facial Landmark Analysis With a 140,000-Way Classifier (AAAI 2018 · similarity 0.87)
- Generalizable Face Landmarking Guided by Conditional Face Warping (CVPR 2024 · similarity 0.85)
- SCE-MAE: Selective Correspondence Enhancement with Masked Autoencoder for Self-Supervised Landmark Estimation (CVPR 2024 · similarity 0.85)
- FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learning (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.