Research map / Image generation / Editing and style transfer
UniHuman: A Unified Model For Editing Human Images in the Wild
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniHuman, a unified Stable Diffusion-based model that jointly performs human reposing, virtual try-on, and text-guided clothing editing, using a pose-warping module, human visual encoders, and a new 400K-image LAION-derived training set.
Problem
Human image editing tasks (reposing, virtual try-on, text-driven clothing editing) are usually handled separately and generalize poorly to real-world, in-the-wild humans with diverse poses, backgrounds, and age groups.
Approach
A single diffusion model that conditions on a training-free pose-warped texture (dense UV or sparse keypoint perspective warping) plus part-level DINOv2/CLIP features, with an attention loss to separate visible-pixel and part-feature sources, and is trained jointly on reposing, try-on, and a curated 400K in-the-wild human dataset.
Key results
- On out-of-domain WPose reposing at 512x512, UniHuman reduces FID from 58 (DisCo) to 27.7.
- In user studies on WPose, UniHuman was preferred in 84.3% (pose accuracy) and 87.6% (texture consistency) of cases against PIDM.
- Virtual try-on out-of-domain WVTON FID improves from 147 (LaDI-VTON) to 128 at 512x512.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add pose-warped texture (ablation) | UniHuman · WPose, 256x256 | FID: 27.571 -> 31.352 | Helped |
| add the pose-warped texture attention loss (ablation) | UniHuman · WPose, 256x256 | FID: 27.571 -> 30.267 | Helped |
| add LH-400K training data (ablation) | UniHuman · WPose, 256x256 | FID: 27.571 -> 37.826 | Helped |
| add virtual try-on training data to reposing training (ablation) | UniHuman · WPose, 256x256 | FID: 27.571 -> 42.587 | Helped |
| add pose-warped texture (ablation) | UniHuman trained without LH-400K · WPose, 256x256 | FID: 37.826 -> 47.736 | Helped |
| use joint multi-task UniHuman rather than task-specific baselines | human reposing · WPose, 512x512 | FID: 58.331 -> 27.748 | Helped |
| use UniHuman rather than LaDI-VTON | virtual try-on · WVTON, 512x512 | FID: 147.375 -> 127.866 | Helped |
| use UniHuman rather than HR-VITON | virtual try-on · WVTON, 512x512 | FID: 159.553 -> 127.866 | Helped |
Surprising findings
- Removing the 400K LH-400K dataset yields comparable or slightly better in-domain SSIM and LPIPS, while out-of-domain metrics degrade substantially.
Limitations
- Out-of-domain evaluation relies on only 2K newly collected pairs, and user studies cover limited sample sizes.
- Text-manipulation evaluation is qualitative only, with limited comparison to baselines.
Experimental setup
Datasets: LAION-400M LH-400K DeepFashion DressCode VITON-HD TikTok
Benchmarks: DeepFashion VITON-HD DressCode WPose WVTON
Metrics: FID SSIM LPIPS KID Masked-SSIM Masked-LPIPS user preference
Baselines: NTED PIDM UPGPT DisCo HR-VITON GP-VTON LaDI-VTON EditAnything
Builds on: Stable Diffusion DINOv2 CLIP PIDM LaDI-VTON
Closest papers by meaning
- UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editing (CVPR 2025 · similarity 0.87)
- UltraEdit: Instruction-based Fine-Grained Image Editing at Scale (NeurIPS 2024 · similarity 0.86)
- Learning Action and Reasoning-Centric Image Editing from Videos and Simulation (NeurIPS 2024 · similarity 0.86)
- FashionTailor: Controllable Clothing Editing for Human Images with Appearance Preserving (AAAI 2025 · similarity 0.86)
- ImgEdit: A Unified Image Editing Dataset and Benchmark (NeurIPS 2025 · similarity 0.86)
- AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.