Research map / Video / Video generation
TV2TV: A Unified Framework for Interleaved Language and Video Generation
Xiaochuang Han, Youssef Emad, Melissa Hall, John Nguyen, Karthik Padthe, Liam Robbins, Amir Bar, Delong Chen, Michal Drozdzal, Maha Elbayad, Yushi Hu, Shang-Wen Li and 5 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| interleave planning text with video chunks instead of generating no text (T2V) (ablation) | TV2TV 3B-MoT · CS:GO pairwise human visual quality | preference rate (%): 91% vs. 1% | Improved |
| interleave text with video chunks instead of generating the full text plan first (Think2V) (ablation) | TV2TV 3B-MoT · CS:GO pairwise human visual quality | preference rate (%): 48% vs. 38% | Improved |
| interleave text with video chunks instead of think-then-act (ablation) | TV2TV 3B-MoT · CS:GO fine-grained controllability | instruction following accuracy: 19 point improvement | Improved |
| interleave text with video chunks (vs Think2V) for the backward and reload interventions (ablation) | TV2TV 3B-MoT · CS:GO interventions | intervention correctness | Mixed |
| interleave text with video chunks (vs Think2V) for left-click and jump interventions (ablation) | TV2TV 3B-MoT · CS:GO interventions | intervention correctness | Improved |
| interleave text with video chunks, assessed on visual quality after manual interventions (ablation) | TV2TV 3B-MoT · CS:GO interventions | visual quality | Improved |
| evaluate on long (64 s) rather than short videos | TV2TV 3B-MoT · CS:GO pairwise human visual quality | preference rate | Improved |
| train on interleaved sports captions instead of T2V on the same data (ablation) | TV2TV 8B-MoT · Sports pairwise human eval | win rate (%): 54.0% vs. 34.7% | Improved |
| train on interleaved sports captions instead of Think2V on the same data (ablation) | TV2TV 8B-MoT · Sports pairwise human eval | win rate (%): 53.3% vs. 41.3% | Improved |
| compare TV2TV on sports to Cosmos2 variants | TV2TV 8B-MoT · Sports pairwise human eval, visual quality | visual quality | Improved |
| compare TV2TV on sports to WAN 2.2 5B | TV2TV 8B-MoT · Sports pairwise human eval, visual quality | visual quality | Hurt |
| compare TV2TV on sports to MAGI-1 variants | TV2TV 8B-MoT · Sports pairwise human eval, visual quality | visual quality | No clear effect |
Closest papers by meaning
- Mimir: Improving Video Diffusion Models for Precise Text Understanding (CVPR 2025 · similarity 0.90)
- T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation (CVPR 2025 · similarity 0.89)
- OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation (ICLR 2025 · similarity 0.88)
- VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis (ICLR 2025 · similarity 0.88)
- OmniViD: A Generative Framework for Universal Video Understanding (CVPR 2024 · similarity 0.88)
- PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
