Research map / Video / Video generation
SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a 0.6B UNet text-to-video model for phones by pruning Stable Diffusion v1.5, searching temporal layers with latency/memory constraints, and distilling to 4 steps via latent adversarial fine-tuning, generating a 5-second video in about 5 seconds on iPhone 16 Pro Max.
Problem
Video diffusion models are too large and slow to run on mobile devices, as they require server GPUs and minutes per video.
Approach
Start from a pruned compact image UNet, search over the type, position, and number of temporal layers using on-device latency and memory profiles, then apply a tailored latent adversarial fine-tuning with a spatial-temporal discriminator head to reduce sampling to 4 steps without classifier-free guidance.
Key results
- Achieves VBench total score 81.14 with 0.6B params and 4 steps, above CogVideoX-2B (80.91) and OpenSora-V1.2 (79.76) and the 4-step AnimateLCM (79.42).
- Generates a 5-second video on iPhone 16 Pro Max in about 5 seconds, with 0.47 s on A100 versus 31 s for OpenSora-V1.2 and 54 s for CogVideoX-2B.
- User study win rates of 44.4% prompt alignment, 48.3% aesthetics, 49.3% motion, versus 30.7% and 30.5% for CogVideoX-2B on the same criteria.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use joint image-video training instead of video-only training (ablation) | SnapGen-V · VBench | total score: −2.54 | Helped |
| Use ×2 temporal layers (ablation) | SnapGen-V · VBench | total score: +0.27 | Helped |
| Use ×4 temporal layers (ablation) | SnapGen-V · VBench | total score: +0.31 | Helped |
| Use ×0.5 temporal layers (ablation) | SnapGen-V · VBench | total score: −1.58 | Hurt |
| Retain temporal layers in downsample blocks (ablation) | SnapGen-V · VBench | total score | Helped |
| Retain bottleneck temporal layers (ablation) | SnapGen-V · VBench | total score | Helped |
| Retain temporal layers in upsample blocks (ablation) | SnapGen-V · VBench | total score | Helped |
| Use unified spatial-temporal discriminator heads instead of separate spatial and temporal heads (ablation) | SnapGen-V adversarial fine-tuning · VBench | semantic score: 64.25 vs. 69.01 | Helped |
| Add image data during adversarial fine-tuning (ablation) | SnapGen-V · VBench | total score: 80.69 vs. 81.14 | Helped |
| Remove adversarial loss (ablation) | SnapGen-V · VBench | total score: 80.69 vs. 59.44 | Hurt |
| Set discriminator noise-distribution parameter m to −2 (ablation) | SnapGen-V adversarial fine-tuning · VBench | total score: 81.14 vs. 81.23 | Helped |
| Set discriminator noise-distribution parameter m to 0 or 1 (ablation) | SnapGen-V adversarial fine-tuning · VBench | total score: 81.14 vs. 79.24; 81.14 vs. 74.05 | Hurt |
Surprising findings
- Scaling up the temporal layers 2x gives only marginal gains (total +0.27) while doubling latency and hitting iPhone memory bounds.
- Including image data in adversarial fine-tuning slightly lowers the VBench quality score but substantially improves multi-object semantic performance.
Limitations
- Uses a public 4-channel VAE; the authors note more latent channels would help reconstruction details.
- Only conventional whole-clip text-to-video generation; streamed real-time generation is left for future work.
- Single-step or 1-2 step generation performs poorly, with 1-step output degrading sharply.
Experimental setup
Datasets: high-quality synthetic image datasets video dataset (unspecified) VBench prompts Movie Gen Bench prompts
Benchmarks: VBench Movie Gen Bench
Metrics: VBench total score quality score semantic score latency on A100 latency on iPhone 16 Pro Max user win rate dynamic degree motion smoothness
Baselines: OpenSora-V1.2 CogVideoX-2B CogVideoX-5B AnimateDiff-V2 AnimateLCM T2V-Turbo VideoCrafter-2.0 Pyramid Flow Kling Emu3 ModelScope SVD
Builds on: Stable Diffusion v1.5 SF-V Rectified Flow AnimateDiff OpenSora VAE
Closest papers by meaning
- MobileWan: Closing the Quality Gap for Mobile Video Diffusion (NeurIPS 2026 · similarity 0.90)
- Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis (CVPR 2024 · similarity 0.89)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.88)
- OSV: One Step is Enough for High-Quality Image to Video Generation (CVPR 2025 · similarity 0.88)
- SF-V: Single Forward Video Generation Model (NeurIPS 2024 · similarity 0.88)
- AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.