Research map / Image generation / Editing and style transfer
StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts a pre-trained Stable Diffusion model into an end-to-end latent-space virtual try-on system using zero cross-attention blocks for clothing-body semantic correspondence, plus augmentation and an attention total variation loss.
Problem
Image-based virtual try-on needs to preserve clothing details while generalizing to arbitrary person images with complex backgrounds, which GAN-based and prior diffusion-based methods struggle with.
Approach
Condition the pre-trained diffusion U-Net on a trainable copy of the SD encoder's clothing features via zero-initialized cross-attention blocks, so warping happens inside the latent space without an external warping network; sharpen correspondence with random-shift augmentation and an attention total variation loss.
Key results
- On VITON-HD single-dataset unpaired evaluation, StableVITON achieves FID 8.698 and KID 0.88, the best among compared methods (DCI-VTON FID 8.754).
- In cross-dataset evaluation (VITON-HD to SHHQ-1.0), StableVITON reports FID 23.531 vs. 24.850 for DCI-VTON.
- User study with 40 participants found StableVITON preferred in most criteria, especially in cross-dataset settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply RePaint during inference | StableVITON · VITON-HD / VITON-HD | FID: 8.698 -> 8.233 | Helped |
| Replace ControlNet conditioning with zero cross-attention blocks (ablation) | StableVITON on VITON-HD · VITON-HD | LPIPS: 0.1157 -> 0.0851 | Helped |
| Replace ControlNet-W conditioning with zero cross-attention blocks (ablation) | StableVITON on VITON-HD · VITON-HD | LPIPS: 0.1124 -> 0.0851 | Helped |
| Add augmentation (ablation) | StableVITON on VITON-HD · VITON-HD | LPIPS: 0.0969 -> 0.0851 | Helped |
| Add attention total variation loss (ablation) | StableVITON with augmentation on VITON-HD · VITON-HD | LPIPS: 0.0851 -> 0.042 | Helped |
| Add attention total variation loss (ablation) | StableVITON with augmentation on VITON-HD · VITON-HD | FID: 8.744 -> 8.698 | Helped |
Surprising findings
- ControlNet with warped clothing (ControlNet-W) did not improve fine clothing detail over plain ControlNet, indicating sensitivity to misalignment in additive conditioning.
Limitations
- Paired-setting SSIM is lower than some baselines without RePaint, attributed to autoencoder reconstruction error of the agnostic map.
- Clothing-detail preservation depends on attention sharpness; without the TV loss, color and logo inaccuracies remain.
Experimental setup
Datasets: VITON-HD DressCode SHHQ-1.0
Benchmarks: VITON-HD DressCode upper-body SHHQ-1.0
Metrics: SSIM LPIPS FID KID user preference
Baselines: VITON-HD HR-VITON GP-VTON LADI-VTON DCI-VTON Paint-by-Example ControlNet
Builds on: Stable Diffusion ControlNet Paint-by-Example RePaint
Closest papers by meaning
- CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models (ICLR 2025 · similarity 0.83)
- Latent Diffusion-Enhanced Virtual Try-On via Optimized Pseudo-Label Generation (AAAI 2025 · similarity 0.81)
- Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting (CVPR 2024 · similarity 0.81)
- Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.81)
- Continuous Latent Diffusion Language Model (NeurIPS 2026 · similarity 0.81)
- Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis (CVPR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.