Research map / Image generation / Editing and style transfer
Cascaded Diffusion Models for Virtual Try-On: Improving Control and Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CDM-VTON, a cascade of a multi-conditioned diffusion model (MC-DM, with a multi-control integration module over a frozen Paint-by-Example U-Net) and a DINO-V2-conditioned super-resolution diffusion model (SR-DM) to produce detail-preserving virtual try-on images up to 2K.
Problem
Virtual try-on diffusion methods struggle to preserve complex garment features (text, patterns, uncommon styles) and are limited to about 1K output resolution.
Approach
Injects openpose and densepose ControlNet control vectors into a frozen denoising U-Net via trainable openpose/densepose attention layers (multi-control integration module), and adds a separate ControlNet-based SR diffusion model that uses DINO-V2 patch features of the low-res try-on image to upsample it with garment fidelity.
Key results
- On VITON-HD at 512x384, the method reaches SSIM 0.896, LPIPS 0.0636, FID 6.062, KID 0.58, beating the prior best FID of IDM-VTON (6.292) and StableVITON SSIM (0.888).
- With SR-DM for 4x upsampling to 2K, MC-DM+SR-DM achieves FID 4.386 and KID 0.37 on VITON-HD, better than MC-DM+DiffBIR (FID 4.859) and IDM-VTON+SR-DM (FID 4.632).
- Ablation: the full model with MIM improves FID from 11.939 (bare Paint-by-Example) to 6.062, while using both control branches without MIM gives FID 7.533.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CDM-VTON instead of StableVITON | VITON-HD · VITON-HD | SSIM: 0.888 -> 0.896 | Helped |
| use CDM-VTON instead of StableVITON | VITON-HD · VITON-HD | LPIPS: 0.0732 -> 0.0636 | Helped |
| use CDM-VTON instead of IDM-VTON | VITON-HD · VITON-HD | FID: 6.292 -> 6.062 | Helped |
| use CDM-VTON instead of CAT-DM | DressCode-Lower · DressCode-Lower | LPIPS: 0.0621 -> 0.0761 | Hurt |
| add Openpose and Densepose control branches and MIM (ablation) | Bare Model · VITON-HD | SSIM: 0.802 -> 0.896 | Helped |
| add Openpose and Densepose control branches and MIM (ablation) | Bare Model · VITON-HD | FID: 11.939 -> 6.062 | Helped |
| add the Densepose ControlNet branch (ablation) | MC-DM with Openpose control · VITON-HD | SSIM: 0.849 -> 0.878 | Helped |
| add the Densepose ControlNet branch (ablation) | MC-DM with Openpose control · VITON-HD | FID: 8.887 -> 7.533 | Helped |
| add MIM to both ControlNet branches (ablation) | MC-DM with Openpose and Densepose controls · VITON-HD | SSIM: 0.878 -> 0.896 | Helped |
| add MIM to both ControlNet branches (ablation) | MC-DM with Openpose and Densepose controls · VITON-HD | FID: 7.533 -> 6.062 | Helped |
| use SR-DM instead of DiffBIR for super-resolution (ablation) | MC-DM · VITON-HD | SSIM: 0.847 -> 0.851 | Helped |
| use SR-DM instead of DiffBIR for super-resolution (ablation) | MC-DM · VITON-HD | LPIPS: 0.0912 -> 0.0857 | Helped |
Surprising findings
- Table 1 shows IDM-VTON has the best FID among baselines but a lower SSIM than StableVITON, indicating metric trade-offs across baselines.
Limitations
- Evaluation limited to two datasets (VITON-HD, DressCode) with standard metrics and no human study.
- Baselines at 2K rely on DiffBIR post-hoc upsampling, so the comparison of end-to-end resolution is not fully matched.
Experimental setup
Datasets: VITON-HD DressCode
Benchmarks: VITON-HD DressCode
Metrics: SSIM LPIPS FID KID
Baselines: HR-VITON Paint-by-Example LaDI-VTON DCI-VTON StableVITON CAT-DM IDM-VTON StableSR DiffBIR SUPIR
Builds on: Paint-by-Example ControlNet IRControlNet CAT-DM Stable Diffusion
Closest papers by meaning
- Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On (CVPR 2024 · similarity 0.93)
- CAT-DM: Controllable Accelerated Virtual Try-on with Diffusion Model (CVPR 2024 · similarity 0.91)
- M&M VTO: Multi-Garment Virtual Try-On and Editing (CVPR 2024 · similarity 0.91)
- Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.90)
- ITA-MDT: Image-Timestep-Adaptive Masked Diffusion Transformer Framework for Image-Based Virtual Try-On (CVPR 2025 · similarity 0.89)
- OOTDiffusion: Outfitting Fusion Based Latent Diffusion for Controllable Virtual Try-On (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.