Research map / Image generation / Editing and style transfer
AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
AnyFit is a diffusion-based virtual try-on system that supports multiple garments via parallel-attention HydraNet branches, initializes from merged SDXL weights, and uses parsing-free mask augmentation for robust fitting.
Problem
Image-based virtual try-on struggles with high-fidelity, robust fitting across scenarios and cannot handle combinations of multiple garments (e.g., tops and bottoms) in one try-on.
Approach
Hydra Encoding/Fusion blocks parallelize self-attention per garment condition within a shared HydraNet whose features are concatenated into MainNet's attention; Prior Model Evolution merges SDXL base, inpainting and DreamShaper weight increments with greedy-searched coefficients; Adaptive Mask Boost trains on parsing-free elongated OpenPose masks and elongates masks at inference based on garment a
Key results
- On VITON-HD, AnyFit reaches FID 8.60 and KID 0.55 versus 8.76/0.92 for DCI-VTON (best prior), with SSIM 0.893.
- On DressCode-multiple, AnyFit achieves FID 20.43 and KID 7.10, beating VTON-concat (21.11/7.30) and IDM-2Stage (21.47/7.85) at about 9% inference-time increase.
- On the proprietary dataset, AnyFit (proprietary) gets FID 43.97 and KID 0.69 versus 48.76/4.35 for IDM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AnyFit instead of DCI-VTON | VITON-HD try-on · VITON-HD | SSIM: 0.881 -> 0.893 | Helped |
| Use AnyFit instead of IDM | DressCode try-on · DressCode | FID: 5.32 -> 4.51 | Helped |
| Use AnyFit instead of VTON-concat | multi-garment try-on · DressCode-multiple | FID: 21.11 -> 20.43 | Helped |
| Use AnyFit trained on proprietary data instead of IDM | single-garment try-on · Proprietary dataset | FID: 48.76 -> 43.97 | Helped |
| Add Hydra Blocks (ablation) | AnyFit multi-garment try-on · DressCode-multiple | FID: 22.48 -> 20.43 | Helped |
| Add Hydra Blocks (ablation) | AnyFit multi-garment try-on · DressCode-multiple | KID: 8.02 -> 7.10 | Helped |
| Add Prior Model Evolution (ablation) | AnyFit multi-garment try-on · DressCode-multiple | FID: 21.35 -> 20.43 | Helped |
| Add Prior Model Evolution (ablation) | AnyFit multi-garment try-on · DressCode-multiple | KID: 7.58 -> 7.10 | Helped |
| Add Adaptive Mask Boost (ablation) | AnyFit single-garment try-on · Proprietary dataset | FID: 44.75 -> 43.97 | Helped |
| Add Adaptive Mask Boost (ablation) | AnyFit single-garment try-on · Proprietary dataset | LPIPS: 0.183 -> 0.181 | Helped |
| Add Prior Model Evolution (ablation) | AnyFit single-garment try-on · Proprietary dataset | FID: 45.01 -> 43.97 | Helped |
| Add Prior Model Evolution (ablation) | AnyFit single-garment try-on · Proprietary dataset | KID: 1.52 -> 0.69 | Helped |
Surprising findings
- On VITON-HD, DCI-VTON has better LPIPS (0.072) than AnyFit (0.075), so AnyFit does not win on every metric.
- Prior Model Evolution improves the model before any training, but the authors note that SDXL-inpainting degrades text-to-image capability relative to SDXL base.
Limitations
- Instability in generating hands with complex structures, inherited from large text-image models.
- Text control over try-on style is initial and unstable.
- Multi-garment flat-lay pairs were never seen in training; robustness to them is observed only qualitatively.
Experimental setup
Datasets: VITON-HD DressCode proprietary e-commerce dataset DressCode-multiple
Benchmarks: VITON-HD DressCode DressCode-multiple proprietary e-commerce dataset
Metrics: LPIPS SSIM FID KID
Baselines: HR-VITON LADI-VTON DCI-VTON StableVTON GP-VTON OOTDiffusion IDM Paint-by-Example IDM-2Stage VTON-concat
Builds on: SDXL ReferenceNet IP-Adapter OOTDiffusion Stable Diffusion
Closest papers by meaning
- AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models (CVPR 2025 · similarity 0.89)
- IMAGDressing-v1: Customizable Virtual Dressing (AAAI 2025 · similarity 0.88)
- OOTDiffusion: Outfitting Fusion Based Latent Diffusion for Controllable Virtual Try-On (AAAI 2025 · similarity 0.88)
- Enhancing Virtual Try-On with Synthetic Pairs and Error-Aware Noise Scheduling (CVPR 2025 · similarity 0.88)
- Virtual Try-On: Real-Time Interactive Hybrid Network with High-Fidelity (AAAI 2024 · similarity 0.88)
- M&M VTO: Multi-Garment Virtual Try-On and Editing (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.