Research map / Image generation / Text-to-image diffusion
Align Your Latents: High-Resolution Video Synthesis With Latent Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add temporal alignment layers to the pretrained image generator | Video LDM · RDS | FVD: 478 -> 389 | Helped |
| Add crowdedness and day/night conditioning | Video LDM · RDS | FVD: 389 -> 356 | Helped |
| Use context guidance (ablation) | Video LDM · RDS | FVD: 534.17 -> 508.82 | Helped |
| Use context guidance (ablation) | Video LDM · RDS | FID: 48.26 -> 54.16 | Hurt |
| Pretrain the spatial image model and train only temporal layers, rather than training end-to-end on videos (ablation) | Video LDM · RDS | FVD: 1155.10 -> 534.17 | Helped |
| Use 3D convolutional temporal layers rather than attention-only temporal layers (ablation) | Video LDM · RDS | FVD: 704.41 -> 534.17 | Helped |
| Temporally fine-tune the upsampler instead of upsampling frames independently (ablation) | Video LDM upsampler · RDS video upsampling | FVD: 165.98 -> 45.39 | Helped |
| Temporally fine-tune the upsampler instead of upsampling frames independently (ablation) | Video LDM upsampler · RDS video upsampling | FID: 19.71 -> 19.85 | No clear effect |
| Video-fine-tune the decoder (ablation) | LDM autoencoder · RDS decoder reconstruction | FVD: 390.88 -> 32.94 | Helped |
| Video-fine-tune the decoder (ablation) | LDM autoencoder · RDS decoder reconstruction | FID: 7.61 -> 9.17 | Hurt |
| Use Video LDM instead of Make-A-Video | Text-to-video generation · UCF-101 | IS: 33.00 -> 33.45 | Helped |
| Use Video LDM instead of Make-A-Video | Text-to-video generation · UCF-101 | FVD: 367.23 -> 550.61 | Hurt |
Closest papers by meaning
- MeDM: Mediating Image Diffusion Models for Video-to-Video Translation with Temporal Correspondence Guidance (AAAI 2024 · similarity 0.92)
- Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition (ICLR 2024 · similarity 0.92)
- Hierarchical Patch Diffusion Models for High-Resolution Video Generation (CVPR 2024 · similarity 0.91)
- Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Search (NeurIPS 2025 · similarity 0.91)
- VideoGuide: Improving Video Diffusion Models without Training Through a Teacher's Guide (CVPR 2025 · similarity 0.91)
- Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.