Research map / Video / Video generation
SwiftTry: Fast and Consistent Video Virtual Try-On with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SwiftTry, a two-stage diffusion video virtual try-on model with temporal attention and ShiftCaching (shifted non-overlapping chunks plus partial computation with masked temporal attention) for faster, consistent long-video try-on, and releases the TikTokDress dataset.
Problem
Video virtual try-on suffers from frame-to-frame inconsistency, and existing chunk-overlap strategies for long videos are computationally expensive; existing datasets (VVT, ViViD) have simple backgrounds and movements.
Approach
Reconceptualizes video try-on as conditional video inpainting built on an image try-on UNet with inserted temporal attention, and replaces overlapping-chunk averaging with ShiftCaching: non-overlapping chunks shifted across denoising steps, plus cached partial computation guarded by Masked Temporal Attention.
Key results
- On VVT, SwiftTry achieves VFID_RN 0.534 and FPS 2.270, versus 1.409 FPS for ViViD re-evaluated, and VFID_I3D 3.589 vs ViViD's 6.788.
- ShiftCaching with 50% partial computation raises inference speed from 1.544 to 2.270 FPS while keeping VFID_I3D around 8.9 on the ablation setting.
- On TikTokDress, SwiftTry reaches LPIPS 0.074 and SSIM 0.888 with 2.270 FPS, versus ViViD's LPIPS 0.129 and SSIM 0.824.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ShiftCaching instead of omitting it (ablation) | SwiftTry · TikTokDress | FPS: 1.177 -> 2.270 | Helped |
| Use ShiftCaching instead of omitting it (ablation) | SwiftTry · TikTokDress | SSIM: 0.891 -> 0.888 | Hurt |
| Use random rather than fixed shifts (ablation) | SwiftTry ShiftCaching · VVT | VFID_I3D: 8.971 -> 8.878 | Helped |
| Partially compute 50% of frames (ablation) | SwiftTry ShiftCaching with fixed shifts · VVT | FPS: 1.544 -> 2.270 | Helped |
| Partially compute 50% of frames (ablation) | SwiftTry ShiftCaching with random shifts · VVT | FPS: 1.544 -> 2.270 | Helped |
| Use random shifts and 50% partial computation (ablation) | SwiftTry ShiftCaching · VVT | VFID_I3D: 8.878 -> 8.938 | Hurt |
| Use Half Attention rather than Full Attention (ablation) | SwiftTry temporal attention · VVT | VFID_I3D: 8.932 -> 8.679 | Helped |
| Replace Half Attention with Quarter Attention (ablation) | SwiftTry temporal attention · VVT | VFID_I3D: 8.679 -> 8.990 | Hurt |
| Replace Half Attention with Causal Attention (ablation) | SwiftTry temporal attention · VVT | VFID_I3D: 8.679 -> 13.520 | Hurt |
| Use 16 rather than 8 inference frames (ablation) | SwiftTry · VVT | LPIPS: 0.062 -> 0.061 | Helped |
| Use 24 rather than 16 inference frames (ablation) | SwiftTry · VVT | LPIPS: 0.061 -> 0.163 | Hurt |
| Use 24 rather than 16 inference frames (ablation) | SwiftTry · VVT | FPS: 1.176 -> 1.723 | Helped |
Surprising findings
- Causal attention in the temporal layers performs much worse (VFID_I3D 13.520) than full or half attention.
- Naively increasing chunk overlap improves consistency only at a large speed cost (FPS drops to 0.104 at S=15).
Limitations
- Relies on manually corrected SAM 2 masks for TikTokDress, which is labor-intensive.
- Inference speed still low (about 2.3 FPS), and results are reported on relatively few test videos.
- Some comparisons rely on reported numbers or re-evaluation of closed-source baselines.
Experimental setup
Datasets: VITON-HD VVT TikTokDress TikTok Dataset
Benchmarks: VVT TikTokDress
Metrics: LPIPS SSIM VFID (I3D) VFID (ResNet) FPS
Baselines: CP-VTON FW-GAN FBAFN MV-TON ClothFormer Tunnel Try-On ViViD WildVidFit StableVITON StableVITON+AnimateAnyone
Builds on: StableVITON AnimateAnyone AnimateDiff Stable Diffusion ViViD
Closest papers by meaning
- Pursuing Temporal-Consistent Video Virtual Try-On via Dynamic Pose Interaction (CVPR 2025 · similarity 0.92)
- InstructVVT: Instruction-Driven Video Virtual Try-On without Auxiliary Spatial Priors (NeurIPS 2026 · similarity 0.89)
- Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On (ICLR 2025 · similarity 0.88)
- Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On (CVPR 2024 · similarity 0.88)
- Cascaded Diffusion Models for Virtual Try-On: Improving Control and Resolution (AAAI 2025 · similarity 0.87)
- BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.