Research map / Video / Video generation
Tell Me What Happened: Unifying Text-Guided Video Completion via Multimodal Masked Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add text guidance to video completion | MMVG · MUGEN TVP | FVD: 370 -> 84.8 | Helped |
| Train one unified model for diverse completion time points | MMVG^U · Kitchen, Flintstones, and MUGEN TVP | FVD: 87.2 -> 80.2; 115.9 -> 108.2; 90.1 -> 84.8 | Helped |
| Add task-specific training after unified training | MMVG^S · TVP, TVR, and TVI on Kitchen, Flintstones, and MUGEN | FVD: Table 1 | Helped |
| Use text plus partial frames rather than partial frames alone | MMVG · TVP human evaluation, all three datasets | ground-truth similarity ranking: Kitchen 1.82 -> 2.15; Flintstones 1.62 -> 2.21; MUGEN 1.55 -> 2.26 | Helped |
| Use temporal-aware VQGAN and VideoSwin decoder instead of neither (ablation) | MMVG · Kitchen TVP | FVD: 81.5 -> 80.2 | Helped |
| Add T-VQ to the model without VideoSwin (ablation) | MMVG · Kitchen TVP | FVD: 81.5 -> 80.7 | Helped |
| Add VideoSwin to the model without T-VQ (ablation) | MMVG · Kitchen TVP | FVD: 81.5 -> 80.8 | Helped |
| Replace parallel decoding with autoregressive decoding (ablation) | T-VQ decoder · Kitchen TVP | FVD: 113.5 -> 80.2 | Helped |
| Increase transformer depth from 12 to 24 layers (ablation) | MMVG · Kitchen TVP | FVD: 99.3 -> 80.2 | Helped |
| Increase codebook size from 1024 to 8192 (ablation) | MMVG · Kitchen TVP | FVD: 80.2 -> 79.9 | No clear effect |
| Use temporal CLIP fine-tuning instead of mean pooling (ablation) | CLIP evaluator · Kitchen, Flintstones, and MUGEN instruction-to-video retrieval | R@1: 11.4 -> 33.6; 73.2 -> 93.4; 11.4 -> 47.2 | Helped |
| Provide additional past frames instead of only the first frame (ablation) | MMVG without instruction · MUGEN TVP | FVD: 400 -> 250 | Helped |
Closest papers by meaning
- VideoMAR: Autoregressive Video Generation with Continuous Tokens (NeurIPS 2025 · similarity 0.85)
- DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation (CVPR 2025 · similarity 0.84)
- StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text (CVPR 2025 · similarity 0.84)
- Enhancing Motion in Text-to-Video Generation with Decomposed Encoding and Conditioning (NeurIPS 2024 · similarity 0.83)
- Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generation (CVPR 2025 · similarity 0.83)
- VISTA-LLAMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokens (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.