Research map / Image generation / Editing and style transfer
Generalizable Face Landmarking Guided by Conditional Face Warping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a generalizable face landmarker (SLPT backbone) from labeled real faces and unlabeled stylized faces by embedding it in a conditional polyharmonic face warper and alternating warper and proximal landmarker optimization.
Problem
Face landmarking fails on stylized faces (caricatures, artworks, avatars) because labeled stylized data is scarce and facial styles are diverse; standard domain adaptation methods give limited gains.
Approach
The face landmarker is the key module of a conditional face warper: its predicted landmarks on a stylized face serve as endpoints of a thin-plate-spline warping field that warps labeled real faces to match the stylized one, producing pseudo landmarks; training alternates warper optimization (gradient-image matching plus landmark warping error) with a proximal landmarker update on pseudo-labeled st
Key results
- In the GZSL (Unseen ArtiFace) setting the method reaches average NME 5.46 versus 7.80-8.07 for the best DA baselines (e.g., ASN 7.80, FDA 8.07).
- On CariFace (target) in the DA setting the method achieves 7.70 NME versus 7.70-8.77 for baselines, with full-test 300W NME 3.20 comparable to the oracle 3.10.
- Alternating optimization outperforms joint optimization on target and unseen domains, which tends to overfit the source domain.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply conditional face warping with alternating optimization rather than the strongest reported DA baseline | SLPT face landmarker, 300W→CariFace; unseen ArtiFace · CariFace | NME: 4.60 -> 7.70 | Helped |
| apply conditional face warping with alternating optimization rather than the strongest reported DA baseline | SLPT face landmarker, 300W→ArtiFace; unseen CariFace · ArtiFace | NME: 12.33 -> 10.93 | Helped |
| use grayscale input and Sobel gradients instead of color input and Sobel gradients (ablation) | Proposed face landmarker · CariFace | NME: 7.863 -> 7.695 | Helped |
| use Sobel gradients instead of Spatial gradients with grayscale input (ablation) | Proposed face landmarker · CariFace | NME: 7.870 -> 7.695 | Helped |
| replace the gradient MSE loss with pixel MSE (ablation) | Proposed face landmarker · DA on Caricature | NME: 7.70 -> 8.26 | Hurt |
| replace the gradient MSE loss with perceptual loss (ablation) | Proposed face landmarker · DA on Caricature | NME: 7.70 -> 8.23 | Hurt |
| remove the gradient MSE loss (ablation) | Proposed face landmarker · DA on Caricature | NME: 7.70 -> 9.48 | Hurt |
| replace SLPT with SBR as the face-landmarker backbone (ablation) | Proposed method, 300W→CariFace; unseen ArtiFace · CariFace | NME: 7.70 -> 8.00 | Hurt |
| replace SLPT with HRNet as the face-landmarker backbone (ablation) | Proposed method, 300W→CariFace; unseen ArtiFace · CariFace | NME: 7.70 -> 7.73 | No clear effect |
| jointly optimize the landmarker and warping field instead of alternating optimization (ablation) | Proposed method, GZSL unseen ArtiFace · ArtiFace | NME | Hurt |
Surprising findings
- Standard domain adaptation methods (RevGrad, CycleGAN, BDL) often fail to improve landmarking on stylized faces, with the landmark loss not dominant in their training.
- Heatmap-based backbones (SBR, HRNet) perform worse than SLPT inside the framework because heatmap decoding breaks end-to-end gradient flow through the warper.
Limitations
- Heatmap-based landmarkers cannot be trained end-to-end through the warper, limiting architecture choice.
- Gains on the source real-face domain are small and the method still lags the oracle on target domains in some cases.
Experimental setup
Datasets: 300W CariFace ArtiFace
Benchmarks: 300W CariFace ArtiFace
Metrics: NME
Baselines: RevGrad CycleGAN BDL ASN FDA SBR HRNet OpenPose SPIGA STAR
Builds on: SLPT WarpGAN RevGrad CycleGAN
Closest papers by meaning
- LaTo: Landmark-tokenized Diffusion Transformer for Fine-grained Human Face Editing (ICLR 2026 · similarity 0.89)
- FaceLift: Semi-supervised 3D Facial Landmark Localization (CVPR 2024 · similarity 0.86)
- Learning Person-Specific Animatable Face Models from In-the-Wild Images via a Shared Base Model (CVPR 2025 · similarity 0.86)
- LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition (CVPR 2024 · similarity 0.85)
- FaceShot: Bring Any Character into Life (ICLR 2025 · similarity 0.85)
- Neural Implicit Morphing of Face Images (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.