Research map / Image generation / Editing and style transfer
BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BooW-VTON, a mask-free virtual try-on diffusion model fine-tuned from IDM-VTON/SDXL on mask-free pseudo-triplets with in-the-wild foreground/background augmentation and a try-on localization attention loss.
Problem
Mask-based virtual try-on inpaints a masked person image, losing spatial and lighting information and causing artifacts in complex in-the-wild scenes with poses and occlusions; prior mask-free methods distill from mask-based teachers and inherit their flaws.
Approach
Build mask-free training pairs from high-quality pseudo person images produced by a refined two-stage mask-based try-on in shop scenes, augment them with synthesized foregrounds and backgrounds, and regularize cross/self-attention with a try-on localization loss that suppresses garment attention outside the try-on region.
Key results
- On VITON-HD paired testing, the full model reaches LPIPS 0.1080, SSIM 0.8618, PSNR 21.80, versus IDM-VTON's 0.1223, 0.8547, 21.06.
- On WildVTON unpaired testing, FID_u drops to 32.53 and KID_u to 4.509, compared with 40.68 / 5.606 for OOTD and 38.77 / 7.686 for IDM-VTON in the same table.
- On DressCode, the method outperforms IDM-VTON across upper, lower, and dress categories on all reported metrics (e.g., dresses LPIPS 0.1044 vs 0.1381).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the mask-based pipeline with mask-free distillation (ablation) | IDM-VTON · WildVTON | FIDu: 38.77 -> 57.52 | Hurt |
| add high-quality pseudo data (ablation) | BooW-VTON base mask-free · VITON-HD | FIDu: 9.467 -> 9.191 | Helped |
| add high-quality pseudo data (ablation) | BooW-VTON base mask-free · StreetVTON | FIDu: 28.81 -> 27.26 | Helped |
| add high-quality pseudo data (ablation) | BooW-VTON base mask-free · WildVTON | FIDu: 57.52 -> 56.14 | Helped |
| add in-the-wild data augmentation (ablation) | BooW-VTON with high-quality pseudo data · VITON-HD | FIDu: 9.191 -> 9.204 | No clear effect |
| add in-the-wild data augmentation (ablation) | BooW-VTON with high-quality pseudo data · StreetVTON | FIDu: 27.26 -> 21.70 | Helped |
| add in-the-wild data augmentation (ablation) | BooW-VTON with high-quality pseudo data · WildVTON | FIDu: 56.14 -> 35.62 | Helped |
| add try-on localization loss (ablation) | BooW-VTON with in-the-wild data augmentation · VITON-HD | FIDu: 9.204 -> 8.809 | Helped |
| add try-on localization loss (ablation) | BooW-VTON with in-the-wild data augmentation · StreetVTON | FIDu: 21.70 -> 20.50 | Helped |
| add try-on localization loss (ablation) | BooW-VTON with in-the-wild data augmentation · WildVTON | FIDu: 35.62 -> 32.53 | Helped |
Surprising findings
- Naively distilling a mask-free student from the mask-based IDM-VTON teacher does not improve in-shop performance and worsens wild-scene FID/KID.
- DCI-VTON achieves better KID than BooW-VTON on StreetVTON, attributed to its mask preserving non-try-on pixels.
Limitations
- Lacks user controllability; missing reference for other garment category yields randomly generated content (e.g., lower body when trying on a T-shirt).
- Slight drop in in-shop performance attributed to slower convergence from strong augmentation.
- Pseudo data depends on quality of the mask-based IDM-VTON teacher output.
Experimental setup
Datasets: VITON-HD DressCode StreetVTON WildVTON
Benchmarks: VITON-HD DressCode StreetVTON WildVTON
Metrics: LPIPS SSIM PSNR FID KID
Baselines: DCI-VTON LaDI-VTON CAT-DM TPD StableVITON OOTD IDM-VTON
Builds on: IDM-VTON SDXL IP-Adapter LayerDiffusion latent diffusion model
Closest papers by meaning
- MV-VTON: Multi-View Virtual Try-On with Diffusion Models (AAAI 2025 · similarity 0.90)
- OmniTry: Virtual Try-On Anything without Masks (NeurIPS 2025 · similarity 0.90)
- Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On (CVPR 2024 · similarity 0.90)
- M&M VTO: Multi-Garment Virtual Try-On and Editing (CVPR 2024 · similarity 0.89)
- Enhancing Virtual Try-On with Synthetic Pairs and Error-Aware Noise Scheduling (CVPR 2025 · similarity 0.89)
- Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.