Research map / Image generation / Editing and style transfer
Stable-Hair: Real-World Hair Transfer via Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Stable-Hair is a two-stage diffusion-based hair transfer framework that first converts source images to bald proxies with a Latent ControlNet-style Bald Converter, then transfers reference hairstyles using a Hair Extractor and Latent IdentityNet on Stable Diffusion v1-5.
Problem
Hair transfer from a reference image to a user face struggles with diverse, intricate real-world hairstyles, and GAN-based methods lose color and identity consistency in non-hair regions.
Approach
Use Stable Diffusion with a two-stage pipeline: a Bald Converter standardizes source images to bald, then a Hair Extractor (trainable U-Net copy with hair cross-attention) injects reference hair features; a Latent ControlNet operating in VAE latent space preserves color and identity better than pixel-space ControlNet.
Key results
- Ours achieves best CLIP-I (0.434), FID (35.128), SSIM (0.680) and IDS (0.779) on 2500 CelebA-HQ pairs versus Barbershop, HairFastGAN, SYH, HairCLIP, HairCLIPv2.
- Latent ControlNet beats pixel ControlNet ablation: FID 35.128 vs 39.236 and PSNR 30.980 vs 29.317.
- User study with 30 volunteers gives Ours 26.8% accuracy, 22.4% preservation, 20.6% naturalness, the top share in each category.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Stable-Hair instead of the strongest baseline | Stable-Hair · CelebA-HQ hair transfer | CLIP-I: 0.431 -> 0.434 | Helped |
| Use Stable-Hair instead of the strongest baseline | Stable-Hair · CelebA-HQ hair transfer | FID: 36.205 -> 35.128 | Helped |
| Use Stable-Hair instead of the strongest baseline | Stable-Hair · CelebA-HQ hair transfer | PSNR: 31.552 -> 30.980 | Mixed |
| Use Stable-Hair instead of the strongest baseline | Stable-Hair · CelebA-HQ hair transfer | SSIM: 0.666 -> 0.680 | Helped |
| Use Stable-Hair instead of the strongest baseline | Stable-Hair · CelebA-HQ hair transfer | IDS: 0.769 -> 0.779 | Helped |
| Use Latent ControlNet instead of pixel-space ControlNet (ablation) | Stable-Hair · CelebA-HQ hair transfer | CLIP-I: 0.423 -> 0.434 | Helped |
| Use Latent ControlNet instead of pixel-space ControlNet (ablation) | Stable-Hair · CelebA-HQ hair transfer | FID: 39.236 -> 35.128 | Helped |
| Use Latent ControlNet instead of pixel-space ControlNet (ablation) | Stable-Hair · CelebA-HQ hair transfer | PSNR: 29.317 -> 30.980 | Helped |
| Use Latent ControlNet instead of pixel-space ControlNet (ablation) | Stable-Hair · CelebA-HQ hair transfer | SSIM: 0.668 -> 0.680 | Helped |
| Use Latent ControlNet instead of pixel-space ControlNet (ablation) | Stable-Hair · CelebA-HQ hair transfer | IDS: 0.770 -> 0.779 | Helped |
| Add video data to image-only training (ablation) | Stable-Hair · CelebA-HQ hair transfer | FID: 33.653 -> 35.128 | Hurt |
| Add video data to image-only training (ablation) | Stable-Hair · CelebA-HQ hair transfer | CLIP-I: 0.431 -> 0.434 | Helped |
Surprising findings
- Training the second stage on randomly inpainted source hairstyles (no bald conversion) failed to learn hair transfer, producing random hairstyles.
- Image-only training gave better FID (33.653) than image+video training, while video data improved pose robustness and other metrics slightly.
Limitations
- May transfer hair accessories to the source image due to training data limitations.
- Relies on synthetic triplets generated by inpainting models and ChatGPT, and on the Bald Converter quality.
- GAN-based baselines and evaluation limited to a 2500-image CelebA-HQ subset plus a 30-person user study.
Experimental setup
Datasets: FFHQ CelebV-HQ NH-FFHQ (generated via HairMapper) CelebA-HQ
Benchmarks: CelebA-HQ
Metrics: CLIP-I FID PSNR SSIM IDS user study accuracy/preservation/naturalness
Baselines: Barbershop HairFastGAN SYH HairCLIP HairCLIPv2 HairMapper
Builds on: Stable Diffusion ControlNet HairFastGAN Hairmapper
Closest papers by meaning
- What to Preserve and What to Transfer: Faithful, Identity-Preserving Diffusion-based Hairstyle Transfer (AAAI 2025 · similarity 0.93)
- HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach (NeurIPS 2024 · similarity 0.89)
- HairDiffusion: Vivid Multi-Colored Hair Editing via Latent Diffusion (NeurIPS 2024 · similarity 0.89)
- DiffLocks: Generating 3D Hair from a Single Image using Diffusion Models (CVPR 2025 · similarity 0.89)
- MonoHair: High-Fidelity Hair Modeling from a Monocular Video (CVPR 2024 · similarity 0.86)
- Perm: A Parametric Representation for Multi-Style 3D Hair Modeling (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.