Research map / Label-efficient and robust learning / Self-supervised and contrastive
SCE-MAE: Selective Correspondence Enhancement with Masked Autoencoder for Self-Supervised Landmark Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCE-MAE, which pretrains an MAE backbone on CelebA and then uses a CARB block with density-peak clustering and a locality-constrained repellence loss to selectively refine correspondences for self-supervised facial landmark matching and detection.
Problem
Self-supervised facial landmark estimation requires locally distinct features, but prior methods use instance-level SSL backbones, memory-heavy hypercolumns, and full pairwise correspondence supervision.
Approach
Replace instance-level SSL with MAE for the first stage, and in the second stage cluster inattentive (non-landmark) tokens via density peak clustering and apply a locality-weighted repellence loss only to attentive tokens and cluster centers, avoiding hypercolumns.
Key results
- Landmark matching mean pixel error on MAFL: SCE-MAE DeiT-B 0.27 (same) and 1.61 (different) vs LEAD 0.48 and 2.50 at 256 dims.
- Landmark detection on all annotations: SCE-MAE DeiT-B achieves 2.07 on MAFL, 5.23 on AFLWM, 3.95 on 300W vs LEAD 2.44, 6.05, 4.87.
- Outperforms prior SSL methods with ~8.6% average relative gain (up to 20.1%) under limited annotations on AFLWM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCE-MAE with DeiT-S instead of the strongest listed baseline, LEAD | Self-supervised landmark matching · MAFL same-identity matching | Mean Pixel Error: 0.51 -> 0.31 | Helped |
| Use SCE-MAE with DeiT-S instead of the strongest listed baseline, CL | Self-supervised landmark matching · MAFL different-identity matching | Mean Pixel Error: 2.19 -> 1.69 | Helped |
| Use SCE-MAE DeiT-B instead of LEAD | Self-supervised landmark detection · MAFL | Inter-ocular Distance (%): 2.44 -> 2.07 | Helped |
| Use SCE-MAE DeiT-B instead of LEAD | Self-supervised landmark detection · AFLWM | Inter-ocular Distance (%): 6.05 -> 5.23 | Helped |
| Use SCE-MAE DeiT-B instead of LEAD | Self-supervised landmark detection · AFLWRO | Inter-ocular Distance (%): 5.71 -> 5.33 | Helped |
| Use SCE-MAE DeiT-B instead of LEAD | Self-supervised landmark detection · AFLWRC | Inter-ocular Distance (%): 5.11 -> 4.60 | Helped |
| Use SCE-MAE DeiT-B instead of DVE | Self-supervised landmark detection · 300W | Inter-ocular Distance (%): 4.58 -> 3.95 | Helped |
| Add clustering to MAE backbone features (ablation) | SCE-MAE without LCR loss · MAFL same-identity matching | Mean Pixel Error: 0.55 -> 0.30 | Helped |
| Add clustering to MAE backbone features (ablation) | SCE-MAE without LCR loss · MAFL different-identity matching | Mean Pixel Error: 3.51 -> 4.04 | Hurt |
| Add LCR loss to the clustered SCE-MAE model (ablation) | SCE-MAE · MAFL same-identity matching | Mean Pixel Error: 0.30 -> 0.27 | Helped |
| Add LCR loss to the clustered SCE-MAE model (ablation) | SCE-MAE · MAFL different-identity matching | Mean Pixel Error: 4.04 -> 1.61 | Helped |
Surprising findings
- Re-annotating the AFLWR test set revealed annotation errors, and all methods achieve lower error on the corrected set.
Limitations
- Performance degrades when poses of reference and test samples are vastly dissimilar or landmarks are severely occluded.
- Requires tuning of attentive rate, number of clusters, and repellence hyperparameters.
Experimental setup
Datasets: CelebA MAFL 300W AFLW (AFLWM) AFLW (AFLWR)
Benchmarks: MAFL AFLWM AFLWRO AFLWRC 300W
Metrics: mean pixel error inter-ocular distance percentage Silhouette Coefficient
Baselines: DVE ContrastLandmark (CL) LEAD
Builds on: Masked Autoencoder (MAE) DVE ContrastLandmark LEAD density peak clustering
Closest papers by meaning
- LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition (CVPR 2024 · similarity 0.85)
- R-MAE: Regions Meet Masked Autoencoders (ICLR 2024 · similarity 0.84)
- Self-Guided Masked Autoencoder (NeurIPS 2024 · similarity 0.84)
- Pose-Guided Self-Training with Two-Stage Clustering for Unsupervised Landmark Discovery (CVPR 2024 · similarity 0.83)
- Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders (AAAI 2024 · similarity 0.83)
- PCP-MAE: Learning to Predict Centers for Point Masked Autoencoders (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.