Research map / Video / Video generation
Photorealistic Video Generation with Diffusion Models
Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, Jose Lezama
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use smaller patch size (1 vs 2 vs 4) (ablation) | W.A.L.T-L · UCF-101 | FVD: 461.8 -> 134.4 -> 60.7 | Improved |
| use local window attention instead of full self-attention (ablation) | W.A.L.T-L · UCF-101 | FVD and speed: FVD 59.9 -> 59.6; 1.20 -> 2.00 sps | Improved |
| add self-conditioning (p_sc 0.0 -> 0.9) (ablation) | W.A.L.T-L · UCF-101 | FVD: 109.9 -> 61.4 | Improved |
| increase AdaLN-LoRA rank r from 2 to 256 (ablation) | W.A.L.T-L · UCF-101 | FVD: 60.7 -> 52.5 | Improved |
| enforce zero terminal SNR noise schedule (ablation) | W.A.L.T-L · UCF-101 | FVD: 91.0 -> 60.7 | Improved |
| add latent normalization (ablation) | W.A.L.T-L · UCF-101 | FVD: 67.9 -> 60.7 | Improved |
| add query-key normalization (ablation) | W.A.L.T-L · UCF-101 | FVD: 59.0 -> 60.7 | No clear effect |
| increase autoencoder latent channels c from 8 to 32 (ablation) | W.A.L.T-L · UCF-101 | FVD: 75.4 -> 83.4 | Hurt |
| add joint image-video training (ablation) | W.A.L.T 419M · UCF-101 zero-shot text-to-video | FVD: 598.8 -> 344.5 | Improved |
| scale base model from 419M to 3B (ablation) | W.A.L.T · UCF-101 zero-shot text-to-video | FVD: 344.5 -> 258.1 | Improved |
| use parameter-matched AdaLN-LoRA-2 XL model instead of separate AdaLN L model (ablation) | W.A.L.T · UCF-101 | FVD: 34.6 -> 36.7 | No clear effect |
| use vanilla classifier-free guidance for ImageNet | W.A.L.T-L · ImageNet 256x256 | FID | No clear effect |
Closest papers by meaning
- VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control (ICLR 2025 · similarity 0.90)
- Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution (CVPR 2024 · similarity 0.90)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.90)
- Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition (ICLR 2024 · similarity 0.90)
- Grid Diffusion Models for Text-to-Video Generation (CVPR 2024 · similarity 0.90)
- Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
