Research map / Video / Video generation
StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StreamingT2V, an autoregressive text-to-long-video generator using a conditional attention module (short-term memory), an appearance preservation module (long-term memory), and randomized blending for autoregressive video enhancement, generating up to 1200 frames without stagnation.
Problem
Text-to-video diffusion models generate short clips (about 16 s); naively extending them autoregressively yields hard scene cuts, video stagnation, or quality degradation over long horizons.
Approach
A CAM module attends to features of the preceding 8-frame chunk via temporal cross-attention (ControlNet-style injection) instead of concatenating the last frame, an APM module injects CLIP features of a fixed anchor frame from the first chunk into text cross-attentions, and a randomized blending of overlapping chunks enables autoregressive enhancement by a high-resolution video enhancer.
Key results
- StreamingT2V achieves MAWE 52.3 on 240-frame videos versus 72.9 for the next best (OpenSoraPlan), a roughly 30% reduction.
- StreamingT2V obtains the highest CLIP score (31.73) among all compared methods, with SCuts 0.04 vs. 0 for FreeNoise, which produces near-static videos.
- SparseCtrl yields 5.48 scene cuts on average versus 0.04 for StreamingT2V, about 100 times more.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use StreamingT2V instead of OpenSoraPlan | Autoregressive long-video generation · 50-prompt test set, 240 frames | MAWE: 72.9 -> 52.3 | Helped |
| Use StreamingT2V instead of FreeNoise | Autoregressive long-video generation · 50-prompt test set, 240 frames | SCuts: 0 -> 0.04 | Hurt |
| Use StreamingT2V instead of FreeNoise | Autoregressive long-video generation · 50-prompt test set, 240 frames | CLIP: 31.55 -> 31.73 | Helped |
| Add randomized blending to merge overlapping enhanced video chunks (ablation) | StreamingT2V video enhancement · Overlapping video chunks | temporal consistency | Helped |
| Replace randomized blending with shared noise only (ablation) | StreamingT2V video enhancement · Overlapping video chunks | temporal consistency | Hurt |
| Replace randomized blending with naive concatenation (ablation) | StreamingT2V video enhancement · Overlapping video chunks | temporal consistency | Hurt |
Surprising findings
- Methods that incorporate CLIP image embeddings (SVD, DynamiCrafter-XL, I2VGen-XL) show low CLIP alignment scores, attributed to domain shift of CLIP on generated frames.
- FreeNoise attains the best (zero) scene-cut score only because it produces near-static videos.
Limitations
- Evaluation relies on a single 50-prompt test set and automatic metrics, with no human evaluation.
- Generalization to DiT architectures (OpenSora) is only shown qualitatively.
Experimental setup
Benchmarks: custom 50-prompt test set
Metrics: MAWE SCuts CLIP score
Baselines: SparseCtrl I2VGen-XL DynamiCrafter-XL SEINE SVD FreeNoise OpenSora v1.2 OpenSoraPlan v1.2
Builds on: ModelScope text-to-video ControlNet SDEdit MS-Vid2Vid-XL DDPM
Closest papers by meaning
- VideoTetris: Towards Compositional Text-to-Video Generation (NeurIPS 2024 · similarity 0.91)
- AdaState: Self-Evolving Anchors for Streaming Video Generation (NeurIPS 2026 · similarity 0.90)
- LVD-2M: A Long-take Video Dataset with Temporally Dense Captions (NeurIPS 2024 · similarity 0.90)
- Grid Diffusion Models for Text-to-Video Generation (CVPR 2024 · similarity 0.90)
- LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexity (CVPR 2025 · similarity 0.90)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.