Research map / Video / Video generation
OSV: One Step is Enough for High-Quality Image to Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OSV, a two-stage training scheme (latent GAN pretraining with LoRA, then adversarial consistency latent distillation) with a DINOv2-based latent discriminator and a Time Travel Sampler, enabling one-step image-to-video generation.
Problem
Video diffusion models require many iterative denoising steps, making generation slow; existing consistency distillation and adversarial distillation methods, transferred from image diffusion, suffer from poor few-step quality, training instability, and high memory cost from decoding latents for the
Approach
Combine GAN-based latent pretraining (LoRA, real data as condition) with adversarial consistency distillation in a second stage, use a frozen DINOv2 discriminator operating directly on upsampled video latents (no VAE decoding), remove classifier-free guidance, and use a multi-step ODE solver ("Time Travel Sampler") to improve distillation accuracy.
Key results
- One-step OSV with Time Travel Sampler achieves FVD 171.15 on OpenVid-1M, beating 8-step AnimateLCM (FVD 184.79).
- One-step OSV reaches FVD 335.36 without TTS versus SF-V one-step FVD 425.68; teacher SVD at 25 steps achieves FVD 156.94.
- Replacing the VAE decoder with upsampled latents reduces average iteration time on H800 from 4.29s to 2.61s and lowers GPU memory versus SF-V (35.8 GB vs 73.5 GB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OSV-TTS for one-step sampling instead of AnimateLCM | Image-to-video generation · OpenVid-1M validation set | FVD: 184.79 -> 171.15 | Helped |
| Use five ODE solver steps instead of one in training (ablation) | OSV · OpenVid-1M validation set | FVD: 332.25 -> 171.15 | Helped |
| Add Stage 1 training (ablation) | OSV · OpenVid-1M validation set | FVD: 221.75 -> 171.15 | Helped |
| Add Stage 2 training with TTS (ablation) | OSV · OpenVid-1M validation set | FVD: 234.13 -> 171.15 | Helped |
| Add TTS when Stage 2 is absent (ablation) | OSV · OpenVid-1M validation set | FVD: 298.35 -> 234.13 | Helped |
| Add adversarial training (ablation) | OSV · OpenVid-1M validation set | FVD: 405.41 -> 171.15 | Helped |
| Remove the VAE decoder from adversarial training (ablation) | OSV · OpenVid-1M validation set | FVD: 232.25 -> 171.15 | Helped |
| Remove CFG instead of using CFG scale 1.5 or 3.0 (ablation) | OSV · OpenVid-1M validation set | FVD: 531.23 -> 335.36; 426.71 -> 335.36 | Helped |
| Use higher-resolution training data in Stage 1 (ablation) | OSV · OpenVid-1M validation set | FVD: 455.35 -> 388.86 | Helped |
| Use mixed-resolution data in Stage 2 instead of 1024×576 throughout (ablation) | OSV · OpenVid-1M validation set | FVD: 171.15 vs 173.14 | No clear effect |
| Use the one-step Time Travel Sampler instead of direct two-step inference | OSV · OpenVid-1M validation set | FVD: 181.95 -> 171.15 | Helped |
Surprising findings
- Adding a VAE decoder to the discriminator (as in ADD) made performance worse (FVD 232.25 vs 171.15), contrary to the expectation that pixel-space discrimination helps.
- Classifier-free guidance hurts distilled one-step models (overexposure), so the method removes CFG entirely.
Limitations
- Distilled model blurs hand movements in human motion; 4 inference steps needed to resolve.
- Time Travel Sampler increases NFE to 2 for the one-step setting, and performance depends on higher training resolution.
- Relies on DINOv2 as feature extractor; stronger backbones are left for future work.
Experimental setup
Datasets: OpenVid-1M
Benchmarks: OpenVid-1M validation set
Metrics: FVD user preference study
Baselines: AnimateLCM SF-V Teacher (SVD with CFG=3.0, 25 steps)
Builds on: Stable Video Diffusion Animate-LCM SF-V ADD LADD
Closest papers by meaning
- SF-V: Single Forward Video Generation Model (NeurIPS 2024 · similarity 0.93)
- Diffusion Adversarial Post-Training for One-Step Video Generation (ICML 2025 · similarity 0.92)
- Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation (NeurIPS 2024 · similarity 0.91)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.91)
- Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model (NeurIPS 2024 · similarity 0.91)
- Towards Precise Scaling Laws for Video Diffusion Transformers (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.