Research map / Image generation / Editing and style transfer
Deformable One-shot Face Stylization via DINO Semantic Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Deformation-aware one-shot face stylization that fine-tunes a StyleGANv2 generator with STN warps using a single real-style image pair, guided by DINO-ViT directional and relative structural losses.
Problem
One-shot face stylization that jointly captures appearance change and structural exaggeration; prior one-shot methods rely on GAN inversion of the style example, which gives unfaithful cross-domain guidance and misses geometric deformation.
Approach
Use a single real-style paired reference instead of a lone style example, and guide generator fine-tuning with DINO token features via a directional deformation loss and a relative structural consistency loss, plus STN-based feature warping and style-mixing color alignment.
Key results
- Ours achieves the best LPIPS-free deformation/identity metrics, e.g., dir-CC 0.196 and dir-ID 0.468 on Example-1 versus best baseline dir-CC 0.157 and dir-ID 0.333 (OneshotCLIP) among one-shot methods.
- User study: ours preferred over MTG 73.0%, JoJoGAN 77.0%, OneshotCLIP 83.7%, and over paired variants MTG-pair 76.7%, JoJoGAN-pair 70.7%, DiFa 71.3%.
- Fine-tuning takes about 10 minutes on a single RTX 3090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DINO-guided deformable stylization instead of single-reference one-shot methods | face stylization · Example-1 | dir-ID: 0.333 -> 0.468 | Helped |
| Use DINO-guided deformable stylization instead of single-reference one-shot methods | face stylization · Example-2 | dir-CC: 0.138 -> 0.194 | Helped |
| Use DINO-guided deformable stylization instead of single-reference one-shot methods | face stylization · Example-3 | dir-CC: 0.136 -> 0.191 | Helped |
| Use DINO-guided deformable stylization instead of paired MTG/JoJoGAN variants | paired one-shot face stylization · Example-1 | dir-CC: 0.195 -> 0.196 | Helped |
| Use DINO-guided deformable stylization instead of paired MTG/JoJoGAN variants | paired one-shot face stylization · Example-2 | dir-CC: 0.174 -> 0.194 | Helped |
| Use DINO-guided deformable stylization instead of paired MTG/JoJoGAN variants | paired one-shot face stylization · Example-3 | LPIPS: 0.379 -> 0.342 | Helped |
| Add the relative structural consistency loss to directional deformation and regularization losses (ablation) | proposed face stylization method · stylized face generation | structural diversity | Helped |
| Add the adversarial loss after the directional deformation and regularization losses (ablation) | proposed face stylization method · stylized face generation | style quality | Helped |
| Add STN blocks to the generator (ablation) | proposed face stylization method · stylized face generation | deformation quality | Helped |
| Add color alignment (ablation) | proposed face stylization method · paired-reference stylization with color gaps | semantic matching | Helped |
Surprising findings
- Using DINO keys (as in prior work) was found to miss some structural semantics in an over-fitting test, so DINO tokens were used instead.
- CLIP and FaRL features were found to be less consistent across real and style faces than DINO features.
Limitations
- Relies on paired real-style references, which are mainly obtained from existing generative stylization models rather than hand-made pairs.
- Current GAN inversion (e4e) is unfaithful for artistic images, which the method works around rather than solves.
Experimental setup
Datasets: FFHQ single real-style reference pair
Benchmarks: three face style examples from Fig. 7 (Example-1/2/3)
Metrics: LPIPS dir-CC dir-ID user preference rate
Baselines: MTG JoJoGAN DiFa OneshotCLIP MTG-pair JoJoGAN-pair
Builds on: StyleGANv2 DINO JoJoGAN e4e StyleCariGAN
Closest papers by meaning
- StyO: Stylize Your Face in Only One-Shot (AAAI 2025 · similarity 0.87)
- LeGO: Leveraging a Surface Deformation Network for Animatable Stylized Face Generation with One Example (CVPR 2024 · similarity 0.86)
- Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and Styles (NeurIPS 2025 · similarity 0.85)
- ReGS: Reference-based Controllable Scene Stylization with Gaussian Splatting (NeurIPS 2024 · similarity 0.85)
- One-Shot Structure-Aware Stylized Image Synthesis (CVPR 2024 · similarity 0.85)
- Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylization (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.