Research map / Image generation / Editing and style transfer
CAT-DM: Controllable Accelerated Virtual Try-on with Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CAT-DM, a diffusion-based virtual try-on model (GC-DM with ControlNet and DINO-V2 garment features plus Poisson blending) that starts sampling from a noised GAN output, reaching high quality in 2 steps.
Problem
Image-based virtual try-on with diffusion models suffers from weak controllability over garment patterns and textures, and requires many denoising steps, limiting real-time use.
Approach
Combine a ControlNet-based garment-conditioned latent diffusion model (PBE backbone, DINO-V2 garment patch features, Poisson blending) with a truncated diffusion start: a pre-trained GAN (GP-VTON) try-on result is noised to step T_trunc=100 and denoised in only 2 DDIM steps.
Key results
- On VITON-HD unpaired setting, CAT-DM reaches FID 8.93 and KID 1.37 versus 11.14 and 3.35 for DCI-VTON, using only 2 sampling steps.
- On VITON-HD paired setting, CAT-DM achieves FID 5.60, KID 0.83, SSIM 0.877, LPIPS 0.0803, outperforming all listed baselines.
- Compared with DCI-VTON's default 50 steps, CAT-DM achieves a 25-fold acceleration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the GAN-initialized truncation acceleration strategy to GC-DM | GC-DM (CAT-DM) · VITON-HD | FIDu: 9.67 -> 8.93 | Helped |
| Add the GAN-initialized truncation acceleration strategy to GC-DM | GC-DM (CAT-DM) · VITON-HD | FIDp: 7.11 -> 5.60 | Helped |
| Use DINO-V2 instead of CLIP for garment feature extraction (ablation) | GC-DM · VITON-HD unpaired | FIDu: 10.21 -> 9.67 | Helped |
| Use DINO-V2 instead of IP-Adapter for garment feature extraction (ablation) | GC-DM · VITON-HD unpaired | FIDu: 11.23 -> 9.67 | Helped |
| Use DINO-V2 instead of SeeCoder for garment feature extraction (ablation) | GC-DM · VITON-HD unpaired | FIDu: 9.94 -> 9.67 | Helped |
| Use Poisson blending instead of direct generation (ablation) | GC-DM with DINO-V2 · VITON-HD unpaired | FIDu: 10.76 -> 9.67 | Helped |
| Use Poisson blending instead of image concatenation (ablation) | GC-DM with DINO-V2 · VITON-HD unpaired | FIDu: 10.57 -> 9.67 | Helped |
| Use Poisson blending instead of direct generation (ablation) | GC-DM with DINO-V2 · VITON-HD paired | SSIMp: 0.835 -> 0.862 | Helped |
| Use Poisson blending instead of image concatenation (ablation) | GC-DM with DINO-V2 · VITON-HD paired | SSIMp: 0.854 -> 0.862 | Helped |
Surprising findings
- GC-DM alone degrades substantially when the number of sampling steps is small, while the GAN-initialized CAT-DM avoids this degradation.
Limitations
- Performance of CAT-DM depends on the quality of the pre-trained GAN-based model.
- Limitations are deferred to the supplement and not described in the main text.
Experimental setup
Datasets: VITON-HD DressCode
Benchmarks: VITON-HD DressCode
Metrics: FID KID SSIM LPIPS
Baselines: VITON-HD HR-VITON GP-VTON PBE MGD LaDI-VTON DCI-VTON
Builds on: ControlNet TDPM DDIM Paint-by-Example
Closest papers by meaning
- Cascaded Diffusion Models for Virtual Try-On: Improving Control and Resolution (AAAI 2025 · similarity 0.91)
- Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On (CVPR 2024 · similarity 0.89)
- OOTDiffusion: Outfitting Fusion Based Latent Diffusion for Controllable Virtual Try-On (AAAI 2025 · similarity 0.88)
- CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models (ICLR 2025 · similarity 0.87)
- ITA-MDT: Image-Timestep-Adaptive Masked Diffusion Transformer Framework for Image-Based Virtual Try-On (CVPR 2025 · similarity 0.87)
- Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.