Research map / Image generation / Editing and style transfer
Zero-Shot Head Swapping in Real-World Scenarios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HID, a zero-shot head-swapping diffusion method that uses an automatically derived inverse-orthogonal mask (IOMask) and a hair injection module to swap full heads onto upper-body images.
Problem
Head swapping (replacing an entire head, including hair and face shape, onto a body image) in real-world full-head and upper-body images, where face-centered cropped methods and their fixed masks fail to blend seamlessly and lose hair details.
Approach
Derives a context-aware edit mask from the component of head-conditioned noise prediction orthogonal to the body-conditioned prediction (IOMask), uses it to blend the inverted body latent during denoising, and adds a hair-embedding fusion module built on PhotoMaker V2 to transfer hairstyle.
Key results
- HID achieves head LPIPS 0.0721 and CLIP-I 0.8512 versus REFace's 0.0770 and 0.7867.
- HID achieves hair LPIPS 0.0596 and CLIP-I 0.8686 versus REFace's 0.0658 and 0.8563.
- HID reports FID 37.19 versus REFace's 40.72.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HID instead of REFace | Zero-shot head swapping · SHHQ-1.0 evaluation (head region) | LPIPS: 0.0770 -> 0.0721 | Helped |
| Use HID instead of REFace | Zero-shot head swapping · SHHQ-1.0 evaluation (head region) | CLIP-I: 0.7867 -> 0.8512 | Helped |
| Use HID instead of REFace | Zero-shot head swapping · SHHQ-1.0 evaluation (hair region) | LPIPS: 0.0658 -> 0.0596 | Helped |
| Use HID instead of REFace | Zero-shot head swapping · SHHQ-1.0 evaluation (hair region) | CLIP-I: 0.8563 -> 0.8686 | Helped |
| Use HID instead of REFace | Zero-shot head swapping · SHHQ-1.0 evaluation | FID: 40.7162 -> 37.1879 | Helped |
| Add the hair injection module (ablation) | HID · Head swapping | hairstyle detail | Helped |
| Add IOMask (ablation) | HID · Head swapping | preservation of body regions outside the head | Helped |
| Use orthogonal filtering in the IO map (ablation) | HID IOMask · Head swapping | mask localization precision | Helped |
Limitations
- Regions beyond the head such as hands, neck, and skin may change unintentionally.
- Hand pose and fine details are not preserved when masked for skin tone matching.
- Only one baseline (REFace) is compared because few zero-shot head-swap methods release code.
Experimental setup
Datasets: SHHQ-1.0
Benchmarks: SHHQ-1.0 evaluation split
Metrics: LPIPS CLIP-I FID
Baselines: REFace
Builds on: PhotoMaker V2 ControlNet DDIM inversion SDXL classifier-free guidance
Closest papers by meaning
- Few-Shot Head Swapping in the Wild (CVPR 2022 · similarity 0.89)
- What to Preserve and What to Transfer: Faithful, Identity-Preserving Diffusion-based Hairstyle Transfer (AAAI 2025 · similarity 0.87)
- FuseAnyPart: Diffusion-Driven Facial Parts Swapping via Multiple Reference Images (NeurIPS 2024 · similarity 0.85)
- Synthetic Prior for Few-Shot Drivable Head Avatar Inversion (CVPR 2025 · similarity 0.84)
- Stable-Hair: Real-World Hair Transfer via Diffusion Model (AAAI 2025 · similarity 0.84)
- MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.