Research map / Vision-language and multimodal / Multimodal LLMs

OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces OpenVTON-Bench, a ~100K high-resolution (up to 1536²) virtual try-on benchmark with a hybrid evaluation protocol combining VLM-as-a-judge scoring and a SAM3-mask multi-scale DINOv3 representation metric, validated against human ratings.

Problem

Virtual try-on evaluation relies on pixel and distribution metrics (SSIM, FID, LPIPS) that miss fine-grained texture and semantic errors, and existing datasets are small, low-resolution, or closed-source.

Approach

Combine a VLM judge scoring five dimensions with a mask-eroded, multi-scale DINOv3 feature similarity that separates boundary errors from interior texture artifacts, and build a large balanced high-resolution dataset via DINOv3 clustering and Gemini captioning.

Key results

• The representation metric (S-bar) achieves Kendall's tau 0.833 with human judgments, versus 0.611 for SSIM and 0.722 for the VLM average score.
• YingHui leads the semantic average (VLM 4.372, human 4.608) and FLUX.2-dev substantially outperforms FLUX.1-Kontext-dev on representation similarity (0.841 vs 0.754).
• Qwen-Editor achieves the best PSNR (26.343) and SSIM (0.905), while YingHui achieves the best FID (7.372).

Surprising findings

• Traditional pixel metrics favor Qwen-Editor on PSNR/SSIM even though visual inspection suggests texture smoothing, while YingHui wins on FID and the semantic metrics.
• The UNO baseline scores low on human identity and texture ratings despite strong VLM scores on some dimensions, showing VLM-human divergence on some dimensions.

Limitations

• Relies on off-the-shelf foundation models (Gemini, DINOv3) for filtering and captioning, which may introduce biases or hallucinations.
• Extreme cases such as complex multi-layer occlusions or acrobatic poses are under-represented.
• Web-collected data includes content with copyright and privacy concerns; released under research-only license.

Experimental setup

Datasets: IMAGDressing-v1 IG Pair Dataset e-commerce crawl (over 3M raw pairs) VITON-HD DressCode

Benchmarks: OpenVTON-Bench

Metrics: Kendall's tau Spearman Pearson PSNR SSIM LPIPS FID S_global S_rep multi-scale VLM five-dimension score

Baselines: OOTD UNO EasyControl FLUX.1-Kontext-dev FLUX.2-dev Nanobanana Qwen-Editor HuiWa YingHui SSIM

Builds on: DINOv3 SAM3 VITON-HD DressCode

Closest papers by meaning

• Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.89)
• M&M VTO: Multi-Garment Virtual Try-On and Editing (CVPR 2024 · similarity 0.89)
• VTON 360: High-Fidelity Virtual Try-On from Any Viewing Direction (CVPR 2025 · similarity 0.88)
• Cascaded Diffusion Models for Virtual Try-On: Improving Control and Resolution (AAAI 2025 · similarity 0.88)
• BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training (CVPR 2025 · similarity 0.88)
• MV-VTON: Multi-View Virtual Try-On with Diffusion Models (AAAI 2025 · similarity 0.88)