Research map / Image generation / Editing and style transfer
Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Texture-Preserving Diffusion (TPD), a warping-free virtual try-on model that concatenates masked person and garment images spatially so SD's self-attention performs texture transfer, plus a decoupled mask prediction step for accurate inpainting masks.
Problem
Warping-free diffusion virtual try-on relies on CLIP or extra image encoders with cross-attention, losing fine-grained garment textures and increasing cost, and coarse inpainting masks remove garment-irrelevant body details, hurting try-on fidelity.
Approach
Concatenate the reference garment and masked person image along the spatial dimension so the Stable Diffusion UNet's existing self-attention blocks transfer textures (Self-Attention-based Texture Transfer), and jointly predict a new-garment clothing mask via a two-stage Decoupled Mask Prediction, combined with the original garment area from human parsing.
Key results
- On VITON-HD, TPD reaches FID 8.54, SSIM 0.90, LPIPS 0.07, the best among compared methods (DCI-VTON FID 8.78).
- On VITON, TPD achieves FID 9.58 and SSIM 0.89, outperforming PF-AFN (FID 10.09).
- Ablation on VITON-HD: removing DMP raises FID from 8.54 to 9.08; removing mask augmentation raises FID to 27.24.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use self-attention texture transfer instead of CLIP image features and cross-attention (ablation) | TPD · VITON-HD | SSIM: 0.85 -> 0.90 | Helped |
| use self-attention texture transfer instead of CLIP image features and cross-attention (ablation) | TPD · VITON-HD | FID: 11.34 -> 8.54 | Helped |
| use self-attention texture transfer instead of CLIP image features and cross-attention (ablation) | TPD · VITON-HD | LPIPS: 0.12 -> 0.07 | Helped |
| concatenate garment and masked person images spatially rather than along the channel dimension (ablation) | TPD · VITON-HD | SSIM: 0.85 -> 0.90 | Helped |
| concatenate garment and masked person images spatially rather than along the channel dimension (ablation) | TPD · VITON-HD | FID: 10.95 -> 8.54 | Helped |
| concatenate garment and masked person images spatially rather than along the channel dimension (ablation) | TPD · VITON-HD | LPIPS: 0.11 -> 0.07 | Helped |
| add Decoupled Mask Prediction instead of using traditional masks (ablation) | TPD · VITON-HD | SSIM: 0.88 -> 0.90 | Helped |
| add Decoupled Mask Prediction instead of using traditional masks (ablation) | TPD · VITON-HD | FID: 9.08 -> 8.54 | Helped |
| add Decoupled Mask Prediction instead of using traditional masks (ablation) | TPD · VITON-HD | LPIPS: 0.08 -> 0.07 | Helped |
| add mask augmentation during training (ablation) | TPD · VITON-HD | SSIM: 0.80 -> 0.90 | Helped |
| add mask augmentation during training (ablation) | TPD · VITON-HD | FID: 27.24 -> 8.54 | Helped |
| add mask augmentation during training (ablation) | TPD · VITON-HD | LPIPS: 0.19 -> 0.07 | Helped |
Limitations
- Evaluated only on images with single-color backgrounds; performance on complex backgrounds unexplored.
- Relies on human parsing, pose and dense pose preprocessing tools.
Experimental setup
Datasets: VITON VITON-HD DeepFashion
Benchmarks: VITON VITON-HD DeepFashion
Metrics: SSIM LPIPS FID
Baselines: CP-VTON ClothFlow ACGPN SDAFN PF-AFN VITON-HD HR-VITON LaDI-VTON DCI-VTON Paint-by-Example
Builds on: Stable Diffusion Paint-by-Example DDPM ControlNet
Closest papers by meaning
- Cascaded Diffusion Models for Virtual Try-On: Improving Control and Resolution (AAAI 2025 · similarity 0.93)
- ITA-MDT: Image-Timestep-Adaptive Masked Diffusion Transformer Framework for Image-Based Virtual Try-On (CVPR 2025 · similarity 0.91)
- Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.91)
- M&M VTO: Multi-Garment Virtual Try-On and Editing (CVPR 2024 · similarity 0.90)
- AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models (CVPR 2025 · similarity 0.90)
- BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.