AI Paper Atlas

Research map / Video / Video generation

Photorealistic Video Generation with Diffusion Models

Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, Jose Lezama

ECCV 2024 · Video › Video generation · approach: Efficient video diffusion models · 259 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use smaller patch size (1 vs 2 vs 4) (ablation)W.A.L.T-L · UCF-101FVD: 461.8 -> 134.4 -> 60.7Improved
use local window attention instead of full self-attention (ablation)W.A.L.T-L · UCF-101FVD and speed: FVD 59.9 -> 59.6; 1.20 -> 2.00 spsImproved
add self-conditioning (p_sc 0.0 -> 0.9) (ablation)W.A.L.T-L · UCF-101FVD: 109.9 -> 61.4Improved
increase AdaLN-LoRA rank r from 2 to 256 (ablation)W.A.L.T-L · UCF-101FVD: 60.7 -> 52.5Improved
enforce zero terminal SNR noise schedule (ablation)W.A.L.T-L · UCF-101FVD: 91.0 -> 60.7Improved
add latent normalization (ablation)W.A.L.T-L · UCF-101FVD: 67.9 -> 60.7Improved
add query-key normalization (ablation)W.A.L.T-L · UCF-101FVD: 59.0 -> 60.7No clear effect
increase autoencoder latent channels c from 8 to 32 (ablation)W.A.L.T-L · UCF-101FVD: 75.4 -> 83.4Hurt
add joint image-video training (ablation)W.A.L.T 419M · UCF-101 zero-shot text-to-videoFVD: 598.8 -> 344.5Improved
scale base model from 419M to 3B (ablation)W.A.L.T · UCF-101 zero-shot text-to-videoFVD: 344.5 -> 258.1Improved
use parameter-matched AdaLN-LoRA-2 XL model instead of separate AdaLN L model (ablation)W.A.L.T · UCF-101FVD: 34.6 -> 36.7No clear effect
use vanilla classifier-free guidance for ImageNetW.A.L.T-L · ImageNet 256x256FIDNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy