Research map / Image generation / Autoregressive and latent generation
ITA-MDT: Image-Timestep-Adaptive Masked Diffusion Transformer Framework for Image-Based Virtual Try-On
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ITA-MDT, a lightweight masked diffusion transformer for image-based virtual try-on with a timestep/complexity-adaptive feature aggregator (ITAFA) and an entropy-based salient region extractor (SRE).
Problem
Image-based virtual try-on needs high garment fidelity, but large pre-trained U-Net diffusion models are heavy and slow, and directly applying masked diffusion transformers loses fine-grained garment detail.
Approach
Adapt MDTv2 to try-on (MDT-IVTON) and condition it on a DINOv2 feature aggregated across layers by learnable weights driven by diffusion timestep and garment complexity (ITAFA), plus a high-resolution crop of the highest-entropy garment region (SRE).
Key results
- On VITON-HD, Base+ITAFA+SRE reaches paired FID 5.462 and LPIPS 0.083 versus 6.000 FID for IDM-VTON (2,991M params, 12.288 s).
- MDT-IVTON with ITAFA+SRE uses 671M parameters and 4.596 s inference, smaller and faster than the compared diffusion baselines.
- On DressCode upper, Base+ITAFA and Base+ITAFA+SRE reach LPIPS 0.034 and SSIM 0.951, beating IDM-VTON's 0.036 and 0.944.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ITAFA to MDT-IVTON (Base) (ablation) | MDT-IVTON (Base) · VITON-HD | LPIPS: 0.099 -> 0.086 | Helped |
| Add ITAFA to MDT-IVTON (Base) (ablation) | MDT-IVTON (Base) · VITON-HD | SSIM: 0.887 -> 0.881 | Hurt |
| Add ITAFA to MDT-IVTON (Base) (ablation) | MDT-IVTON (Base) · VITON-HD | FID (paired/unpaired): 7.773/9.838 -> 5.717/8.879 | Helped |
| Add ITAFA to MDT-IVTON (Base) (ablation) | MDT-IVTON (Base) · DressCode Upper | LPIPS: 0.061 -> 0.034 | Helped |
| Add ITAFA to MDT-IVTON (Base) (ablation) | MDT-IVTON (Base) · DressCode Upper | SSIM: 0.921 -> 0.951 | Helped |
| Add ITAFA to MDT-IVTON (Base) (ablation) | MDT-IVTON (Base) · DressCode Upper | FID (paired/unpaired): 10.300/13.998 -> 5.414/13.111 | Mixed |
| Add SRE to Base+ITAFA (ablation) | MDT-IVTON with ITAFA · VITON-HD | FID (paired/unpaired): 5.717/8.879 -> 5.462/8.676 | Helped |
| Replace SRE with a single high-resolution 448 × 448 × 3 garment image (ablation) | MDT-IVTON with ITAFA · DressCode Upper | FID (paired/unpaired): 5.412/10.069 -> 5.384/12.528 | Mixed |
Surprising findings
- Naively adding ITAFA lowers paired FID on VITON-HD, but the best unpaired FID on DressCode is obtained with ITAFA+SRE rather than the high-resolution variant.
Limitations
- Gains over the baseline are mixed: on VITON-HD the paired FID of the full model is not uniformly best, and the unpaired FID on DressCode upper is still worse than some baselines.
- Baseline numbers were re-measured by the authors from released checkpoints, so comparisons depend on their setup.
- The SRE and ITAFA details rely on heuristic thresholds (e.g., sparsity threshold 0.01) without broad sensitivity analysis in the main text.
Experimental setup
Datasets: VITON-HD DressCode
Benchmarks: VITON-HD DressCode upper
Metrics: LPIPS SSIM FID (paired/unpaired) inference time parameter count
Baselines: LaDI-VTON DCI-VTON StableVITON CAT-DM TPD IDM-VTON
Builds on: Masked Diffusion Transformer (MDTv2) Diffusion Transformer (DiT) Latent Diffusion Models LaDI-VTON evaluation protocol StableVITON data augmentation
Closest papers by meaning
- Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On (CVPR 2024 · similarity 0.91)
- Cascaded Diffusion Models for Virtual Try-On: Improving Control and Resolution (AAAI 2025 · similarity 0.89)
- AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models (CVPR 2025 · similarity 0.89)
- Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.89)
- M&M VTO: Multi-Garment Virtual Try-On and Editing (CVPR 2024 · similarity 0.88)
- MV-VTON: Multi-View Virtual Try-On with Diffusion Models (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.