Research map / Image generation / Editing and style transfer
PICTURE: PhotorealistIC virtual Try-on from UnconstRained dEsigns
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ucVTON, a two-stage Stable Diffusion pipeline that disentangles style (text or image) from texture (full garment, crop, or patch) for photorealistic virtual try-on, using hierarchical balanced CLIP features and position encoding.
Problem
Existing virtual try-on methods are constrained in input types: style is limited to text and texture to small image patches, and full garment images entangle style and texture so they cannot be recombined.
Approach
A two-stage pipeline that first edits a human parsing map to the desired style (text or image), then inpaints textures onto the garment regions using hierarchical, balanced CLIP features from all ViT blocks plus position encoding to handle non-stationary textures.
Key results
- Style prediction accuracy of 92.35% vs 88.87% for Text2Human and 82.75% for FashionTex.
- Texture patch FID 21.25 and KID 0.22 (x1000), outperforming Texture Reformer (28.43 FID) and Paint-by-Example (28.73 FID).
- User studies with 96 participants: method selected best by 71% to 88% of participants across texture and style criteria.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use hierarchical CLIP features from all 8 categories instead of a single CLIP feature (ablation) | ucVTON · Garment texture transfer | FID: 24.57 -> 22.25 | Helped |
| use hierarchical CLIP features from all 8 categories instead of a single CLIP feature (ablation) | ucVTON · Garment texture transfer | KID: 4.57 -> 0.22 | Helped |
| use CLIP features from all 8 categories instead of features from layers 1–2 (ablation) | ucVTON · Garment texture transfer | FID: 22.87 -> 22.25 | Helped |
| use CLIP features from all 8 categories instead of features from layers 1–2 (ablation) | ucVTON · Garment texture transfer | KID: 0.70 -> 0.22 | Helped |
| add two-stage style and texture disentanglement instead of one-stage conditioning (ablation) | ucVTON · Style and texture control | style-texture disentanglement | Helped |
| add hierarchical and balanced CLIP feature extraction (ablation) | ucVTON · Garment texture transfer | texture fidelity | Helped |
| add position encoding to CLIP texture features (ablation) | ucVTON · Non-stationary garment texture transfer | texture correspondence | Helped |
Surprising findings
- Naively replacing text CLIP features with image CLIP features gives little improvement because low-level texture cues wash out through CLIP blocks.
Limitations
- Performance depends on accuracy of human parsing maps, which degrade under low light and severe occlusion.
- Disentangle-and-recompose approach departs from warping paradigm, needing extra logo and texture extraction steps for structured garments.
Experimental setup
Datasets: DeepFashion-Multimodal SHHQ VITON-HD
Benchmarks: DeepFashion-Multimodal SHHQ VITON-HD
Metrics: style prediction accuracy FID KID user study preference
Baselines: Texture Reformer PIDM FashionTex Text2Human Paint-by-Example
Builds on: Stable Diffusion FashionTex Paint-by-Example Text2Human
Closest papers by meaning
- IMAGDressing-v1: Customizable Virtual Dressing (AAAI 2025 · similarity 0.89)
- M&M VTO: Multi-Garment Virtual Try-On and Editing (CVPR 2024 · similarity 0.87)
- Virtual Try-On: Real-Time Interactive Hybrid Network with High-Fidelity (AAAI 2024 · similarity 0.87)
- FashionR2R: Texture-preserving Rendered-to-Real Image Translation with Diffusion Models (NeurIPS 2024 · similarity 0.87)
- DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images (NeurIPS 2026 · similarity 0.86)
- FashionTailor: Controllable Clothing Editing for Human Images with Appearance Preserving (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.