Research map / Video / Video generation
VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Decomposed base+residual noise (shared base noise across frames) vs standard independent noise | VideoFusion vs VDM · UCF101 unconditional 16x64x64 | FVD: 295 -> 139 | Helped |
| Pretrained image DPM as base generator plus decomposed noise (ablation) | VideoFusion · Study on pretraining, UCF101 unconditional | FVD: 183 -> 139 with pretraining | Helped |
| Initialize base generator from pretrained image DPM (vs training from scratch) (ablation) | VideoFusion · UCF101 unconditional | IS: 65.29 -> 71.67 | Helped |
| Freeze pretrained base generator instead of joint finetuning (ablation) | VideoFusion · UCF101 unconditional | FVD: 139 -> 187 | Hurt |
| Remove stop-gradient on base generator during joint training (ablation) | VideoFusion · UCF101 unconditional | FVD: 168 -> 139 with stop gradient | Helped |
| Set lambda to 0.10 (less shared base noise) (ablation) | VideoFusion · UCF101 unconditional | FVD: 139 -> 149 | Hurt |
| Set lambda to 0.75 (more shared base noise) (ablation) | VideoFusion · UCF101 unconditional | FVD: 139 -> 181 | Hurt |
| Lambda 0.25 vs 0.50 (ablation) | VideoFusion · UCF101 unconditional | FVD: FVD 122 vs 139 but IS 69.16 vs 71.67 | Mixed |
| Decomposed diffusion with pretrained base generator vs VDM at same resolution | VideoFusion · UCF101 class-conditioned / Sky / TaiChi | FVD: Sky 183.6->47.0 vs MoCoGAN-HD; TaiChi 94.6->56.4 vs TATS | Helped |
| Shared base noise with smaller residual generator and single base forward pass | VideoFusion vs re-implemented VDM · Efficiency, WebVid-scale base generator | Latency (s): 0.40 -> 0.17 (-57.5%) | Helped |
| Shared base noise with smaller residual generator and single base forward pass | VideoFusion vs re-implemented VDM · Efficiency, WebVid-scale base generator | Memory (GB): 63.82 -> 49.85 (-21.8%) | Helped |
Closest papers by meaning
- Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition (ICLR 2024 · similarity 0.89)
- VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Step (CVPR 2025 · similarity 0.88)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.88)
- Diff-BGM: A Diffusion Model for Video Background Music Generation (CVPR 2024 · similarity 0.88)
- VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models (CVPR 2024 · similarity 0.87)
- Spectral Progressive Diffusion for Efficient Image and Video Generation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.