Research map / Image generation / Fast diffusion sampling

CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models

ICLR 2025 · Image generation › Fast diffusion sampling · approach: Fast diffusion ODE solvers

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

CatVTON concatenates garment and person images spatially into a single simplified Stable Diffusion UNet (no encoders, ReferenceNet, or cross-attention), trains only self-attention layers (49.57M params), and drops pose/parsing preprocessing at inference for efficient virtual try-on.

Problem

Diffusion-based virtual try-on methods require extra encoders (CLIP, DINOv2), ReferenceNet branches, many trainable parameters, and preprocessing such as pose estimation, human parsing, and captioning, which raise training and inference costs.

Approach

Spatially concatenate the garment and person images as input to a single denoising UNet of Stable Diffusion, remove text encoders and cross-attention, fine-tune only self-attention layers for parameter-efficient training, and use only person image plus garment at inference (mask-based or mask-free variants).

Key results

• On VITON-HD paired setting, CatVTON (Inpainting) reaches FID 5.425 and KID 0.411, versus 5.762 FID for the next best (IDM-VTON).
• CatVTON has 899.06M total parameters and 49.57M trainable parameters, with 4.00 GB inference memory at 512x384.
• Training only self-attention layers (49.57M) matches full-UNet training (815.45M trainable) in quality, with a 4.75 IPS training speed versus 3.21 for full UNet.

Measured effects reported in the paper

Surprising findings

• Training only self-attention layers performs on par with or better than full UNet fine-tuning despite training ~6% of the parameters.
• Adding text conditions to the self-attention-only variant degraded performance, indicating text conditions are redundant for image-based try-on.

Limitations

• VAE decoding causes detail loss and color discrepancies, especially at 512x384 resolution.
• Performance depends on accuracy of the provided mask for the mask-based variant.
• Inherits biases and NSFW content exposure from Stable Diffusion v1.5.

Experimental setup

Datasets: VITON-HD DressCode DeepFashion

Benchmarks: VITON-HD DressCode DeepFashion

Metrics: SSIM LPIPS FID KID GFLOPs inference time memory usage parameter count

Baselines: DCI-VTON StableVTON StableGarment MV-VTON GP-VTON LaDI-VTON IDM-VTON OOTDiffusion

Builds on: Stable Diffusion Latent Diffusion Models DREAM InstructPix2Pix classifier-free guidance

Closest papers by meaning

• Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On (CVPR 2024 · similarity 0.89)
• Cascaded Diffusion Models for Virtual Try-On: Improving Control and Resolution (AAAI 2025 · similarity 0.89)
• Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.88)
• On improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models (NeurIPS 2024 · similarity 0.88)
• Diffusion PID: Interpreting Diffusion via Partial Information Decomposition (NeurIPS 2024 · similarity 0.88)
• Plug-and-Play Diffusion Distillation (CVPR 2024 · similarity 0.88)