Research map / Label-efficient and robust learning / Self-supervised and contrastive
PrefAce: Face-Centric Pretraining with Self-Structure Aware Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PrefAce, a self-supervised face-centric video pretraining framework combining landmark-guided multiscale tube self-distillation with a momentum-updated FaceFeat Cache for instance discrimination, evaluated on facial attribute, expression, deepfake, and lip-sync tasks.
Problem
Learning transferable facial video representations without labels, since annotated facial data is scarce and expensive, and prior video face pretraining (MARLIN) tends to learn common face features while missing diverse, discriminative ones.
Approach
Applies self-distillation (teacher-student) to video-level facial pretraining with landmark-guided spatio-temporal tube cropping, and adds a FaceFeat Cache memory dictionary of per-video instance features updated by momentum, with an InfoNCE instance loss, to encourage diverse, instance-specific embeddings.
Key results
- On CelebV-HQ, fine-tuned PrefAce reaches 94.84% appearance and 96.35% action accuracy, beating fine-tuned MARLIN (93.90%, 95.48%).
- On FaceForensics++ deepfake detection, PrefAce fine-tuned achieves 92.30% accuracy vs. 89.43% for MARLIN.
- On LRS2 lip sync, Wav2Lip+ViT+PrefAce improves LSE-D from 7.127 to 7.013 and FID from 3.452 to 3.264 over MARLIN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PrefAce instead of MARLIN | Video facial representation pretraining · CelebV-HQ, Appearance | accuracy: 93.90% →94.84% | Helped |
| Use PrefAce instead of MARLIN | Video facial representation pretraining · MOSEI, Sentiment (2-Class) | accuracy: 73.70% →74.05% | Helped |
| Use PrefAce instead of MARLIN | Video facial representation pretraining · MOSEI, Sentiment (7-Class) | accuracy: 34.63% → 35.03% | Helped |
| Use PrefAce instead of MARLIN | Video facial representation pretraining · MOSEI, Emotion | accuracy: 80.60% → 81.29% | Helped |
| Use PrefAce instead of MARLIN | Deepfake detection · FaceForensics++ (LQ) | accuracy: 89.43% →92.30% | Helped |
| Use PrefAce instead of MARLIN | Wav2Lip + ViT · LRS2 | LSE-D: 7.127 →7.013 | Helped |
| Add multiscale landmark-guided distillation (ablation) | PrefAce · MOSEI | accuracy: 80.25 → 80.61 | Helped |
| Add FaceFeat Cache without landmark distillation (ablation) | PrefAce · MOSEI | accuracy: 80.25 → 80.67 | Mixed |
| Add both landmark distillation and FaceFeat Cache (ablation) | PrefAce · MOSEI | accuracy: 80.25 → 81.29 | Helped |
| Use ViT-B instead of ViT-S (ablation) | PrefAce · MOSEI | accuracy: 80.69 → 81.29 | Helped |
Surprising findings
- Facial region-guided tube masking in MARLIN degrades to whole-face masking under its high mask ratio, losing granularity.
- PrefAce's linear-probed version underperforms MARLIN fine-tuned on some metrics, and the Appearance accuracy of linear-probed PrefAce (92.31%) is below linear-probed MARLIN (91.90%) only slightly higher, while overall LP accuracy (93.93%) is above MARLIN LP (93.57%).
Limitations
- Pretraining relies on a single unlabeled source (YTF) and evaluation depends on comparisons with MARLIN under matched protocols.
- Lip synchronization is evaluated only by plugging the encoder into Wav2Lip, not as a standalone study.
- Linear-probed PrefAce does not consistently beat baselines (e.g., MOSEI emotion LP tables show mixed gains).
Experimental setup
Datasets: YouTube Faces (YTF) CelebV-HQ CMU-MOSEI FaceForensics++ LRS2 IEMOCAP
Benchmarks: CelebV-HQ CMU-MOSEI FaceForensics++ LRS2
Metrics: accuracy AUC LSE-D LSE-C FID
Baselines: MARLIN VideoMAE MViTv1 MViTv2 R3D CAE-LR Wav2Lip LipGAN Speech2Vid AttnWav2Lip
Builds on: DINO MARLIN MoCo VideoMAE Wav2Lip
Closest papers by meaning
- LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition (CVPR 2024 · similarity 0.88)
- Self-Supervised Facial Representation Learning with Facial Region Awareness (CVPR 2024 · similarity 0.88)
- FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learning (CVPR 2025 · similarity 0.86)
- Facial-R1: Aligning Reasoning and Recognition for Facial Emotion Analysis (AAAI 2026 · similarity 0.85)
- FSRT: Facial Scene Representation Transformer for Face Reenactment from Factorized Appearance Head-pose and Facial Expression Features (CVPR 2024 · similarity 0.84)
- Learning Person-Specific Animatable Face Models from In-the-Wild Images via a Shared Base Model (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.