Research map / Image generation / Editing and style transfer
HairDiffusion: Vivid Multi-Colored Hair Editing via Latent Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HairDiffusion, a latent-diffusion (Stable Diffusion inpainting + ControlNet) hair editing framework that separates hair color and hairstyle via Multi-stage Hairstyle Blend, a warping module for color alignment, and a CLIP text encoder fine-tuned on multi-color hair data.
Problem
Hair editing (color and hairstyle, from text or reference images) while preserving identity and background; StyleGAN-based methods struggle with multi-color hair and facial preservation.
Approach
Reframes hair editing as diffusion inpainting with hair-agnostic masks and two proxies (a style proxy from ControlNet-guided hairstyle generation and a color proxy from a warped reference hair), blended at different denoising stages (MHB), plus a HR-VITON-style warping module trained with augmented hair representations and a CLIP text encoder fine-tuned on a multi-color hair dataset.
Key results
- On CelebA-HQ text-driven editing, HairDiffusion reports IDS 0.94, PSNR 33.1, SSIM 0.95, versus HairCLIPv2 at 0.84 / 29.5 / 0.91.
- In self-transfer reconstruction, HairDiffusion reaches PSNR 31.66 and LPIPS 0.07, compared with HairFastGAN's 23.45 PSNR and 0.08 LPIPS.
- In a user study with 20 volunteers, the method was preferred for accuracy and preservation over most baselines, but was slightly behind PowerPaint on preservation and SYH on naturalness.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HairDiffusion instead of HairCLIPv2 for text-driven hair editing | Hair editing on CelebA-HQ · CelebA-HQ test set (2,000 images) | IDS: 0.84 -> 0.94 | Helped |
| Use HairDiffusion instead of HairCLIPv2 for text-driven hair editing | Hair editing on CelebA-HQ · CelebA-HQ test set (2,000 images) | PSNR: 29.5 -> 33.1 | Helped |
| Use HairDiffusion instead of HairCLIPv2 for text-driven hair editing | Hair editing on CelebA-HQ · CelebA-HQ test set (2,000 images) | SSIM: 0.91 -> 0.95 | Helped |
| Use HairDiffusion instead of HairFastGAN | Hair color transfer · Single color transfer | FID: 20.17 -> 20.83 | Hurt |
| Use HairDiffusion instead of HairFastGAN | Hair color transfer · Single color transfer | FID_CLIP: 3.00 -> 5.96 | Hurt |
| Use HairDiffusion instead of HairFastGAN | Self-transfer reconstruction · Reconstruction | LPIPS: 0.08 -> 0.07 | Helped |
| Use HairDiffusion instead of HairFastGAN | Self-transfer reconstruction · Reconstruction | PSNR: 23.45 -> 31.66 | Helped |
| Use HairDiffusion instead of HairFastGAN | Self-transfer reconstruction · Reconstruction | FID: 9.72 -> 5.41 | Helped |
| Use HairDiffusion instead of HairFastGAN | Self-transfer reconstruction · Reconstruction | FID_CLIP: 0.97 -> 0.68 | Helped |
| Add the warping module (ablation) | HairDiffusion · Hair color and hairstyle generation quality | FID: 33.17 -> 20.83 | Helped |
| Add PatchMatch (ablation) | HairDiffusion · Hair color and hairstyle generation quality | FID: 27.74 -> 20.83 | Helped |
| Add bilateral filtering (ablation) | HairDiffusion · Hair color and hairstyle generation quality | FID: 20.85 -> 20.83 | No clear effect |
Surprising findings
- Single-color transfer metrics are weaker than StyleGAN-based methods (FID 20.83 vs 19.65 for CtrlHair), which the authors attribute to the base diffusion model.
- Color balance shifts when color proxies differ strongly from the target image, lowering preservation scores relative to MichiGAN.
Limitations
- Struggles when reference and target hairstyles differ strongly in structure, length, or pose (e.g., ombre from short hair onto long hair, profile views).
- Depends on face/hair segmentation quality; intricate styles such as cornrows are hard to segment.
- Color discrepancies in non-hair attributes (e.g., skin or hair color in background) due to latent conditioning design.
Experimental setup
Datasets: CelebAMask-HQ CelebA-HQ multi-color hairstyle dataset (crawled, 4,625 pairs)
Benchmarks: CelebA-HQ test set
Metrics: IDS PSNR SSIM FID FID_CLIP LPIPS user preference (accuracy, preservation, naturalness)
Baselines: HairCLIPv2 HairCLIP TediGAN DiffCLIP PowerPaint ControlNet-Inpainting MichiGAN Barbershop HairFastGAN CtrlHair
Builds on: Stable Diffusion ControlNet HR-VITON HairCLIP textual inversion
Closest papers by meaning
- What to Preserve and What to Transfer: Faithful, Identity-Preserving Diffusion-based Hairstyle Transfer (AAAI 2025 · similarity 0.89)
- Stable-Hair: Real-World Hair Transfer via Diffusion Model (AAAI 2025 · similarity 0.89)
- DiffLocks: Generating 3D Hair from a Single Image using Diffusion Models (CVPR 2025 · similarity 0.87)
- Perm: A Parametric Representation for Multi-Style 3D Hair Modeling (ICLR 2025 · similarity 0.86)
- Free-Lunch Color-Texture Disentanglement for Stylized Image Generation (NeurIPS 2025 · similarity 0.85)
- Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.