Research map / Video / Video segmentation and tracking
Pursuing Temporal-Consistent Video Virtual Try-On via Dynamic Pose Interaction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPIDM, a latent diffusion video virtual try-on model that adds skeleton-based human and garment pose adapters into spatial and temporal attention plus a temporal regularized attention loss.
Problem
Video virtual try-on must keep garment appearance authentic while adapting to body pose and keeping frames temporally consistent, but existing methods ignore spatiotemporal human-garment pose interactions.
Approach
Injects synchronized human and garment pose embeddings via zero-initialized pose adapters into pose-aware spatial attention and pose-aware temporal attention, combined with temporal-shift attention and a loss that penalizes changes in self-attention maps across consecutive frames.
Key results
- On VVT, DPIDM reaches VFID(I3D) 0.506, a 60.5% relative improvement over GPD-VVTO (1.280).
- On ViViD, DPIDM achieves VFID(I3D) 0.488 versus 1.894 for ViViD (74.2% relative improvement) and SSIM 0.883.
- On VITON-HD unpaired setting, DPIDM reports FID 8.15 and KID 0.32, outperforming the listed baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DPIDM instead of GPD-VVTO | video virtual try-on · VVT test | VFIDI: 1.280 -> 0.506 | Helped |
| use DPIDM instead of ViViD | video virtual try-on · ViViD test | VFIDI: 1.894 -> 0.488 | Helped |
| add pose-aware attention (ablation) | DPIDM · VVT test | VFIDI: 3.451 -> 1.068 | Helped |
| add pose-aware attention (ablation) | DPIDM · VVT test | SSIM: 0.893 -> 0.925 | Helped |
| add temporal-shift attention (ablation) | DPIDM with pose-aware attention · VVT test | VFIDI: 1.068 -> 0.721 | Helped |
| add temporal regularized attention loss (ablation) | DPIDM with pose-aware and temporal-shift attention · VVT test | VFIDI: 0.721 -> 0.506 | Helped |
| add temporal-shift attention (ablation) | DPIDM with pose-aware attention · VVT test | SSIM: 0.925 -> 0.929 | No clear effect |
| add temporal regularized attention loss (ablation) | DPIDM with pose-aware and temporal-shift attention · VVT test | LPIPS: 0.043 -> 0.041 | No clear effect |
Surprising findings
- Pose-aware attention alone gives the largest gain; TSA and TRA mainly improve VFID rather than SSIM/LPIPS.
Limitations
- ViViD has no comparison to GPD-VVTO and used different SD versions, complicating fair comparison
- Garment pose estimator requires manually labeled landmarks
- ViViD test set excludes back-facing segments due to inaccurate masks
Experimental setup
Datasets: VVT ViViD VITON-HD
Benchmarks: VVT ViViD VITON-HD
Metrics: SSIM LPIPS VFID FID KID
Baselines: CP-VTON FW-GAN PBAFN ClothFormer LaDI-VTON StableVITON TunnelTry-on ViViD VITON-DiT GPD-VVTO CAT-DM IDM
Builds on: Stable Diffusion ViViD GPD-VVTO DWPose AnimateDiff
Closest papers by meaning
- SwiftTry: Fast and Consistent Video Virtual Try-On with Diffusion Models (AAAI 2025 · similarity 0.92)
- InstructVVT: Instruction-Driven Video Virtual Try-On without Auxiliary Spatial Priors (NeurIPS 2026 · similarity 0.90)
- Robust-MVTON: Learning Cross-Pose Feature Alignment and Fusion for Robust Multi-View Virtual Try-On (CVPR 2025 · similarity 0.88)
- Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.88)
- FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customization (NeurIPS 2026 · similarity 0.88)
- MV-VTON: Multi-View Virtual Try-On with Diffusion Models (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.