Research map / Video / Video generation
LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LinGen, a linear-complexity text-to-video DiT that replaces self-attention with a MATE block (bidirectional Mamba2 with Rotary-Major Scan and review tokens plus a temporal Swin attention branch) to generate minute-length high-resolution videos on a single GPU.
Problem
Quadratic cost of self-attention in Diffusion Transformers makes high-resolution minute-length text-to-video generation prohibitively expensive, and naive Mamba suffers from an adjacency preservation issue when flattening 3D video tokens.
Approach
Replace DiT self-attention with a linear-complexity MATE block combining a bidirectional Mamba2 branch (with rotating scan orders and pooled review tokens for long-range context) and a shifted-window temporal attention branch that restores spatial and temporal adjacency.
Key results
- LinGen-4B achieves 15x FLOPs reduction and 11.5x latency reduction versus DiT-4B when generating 68s 512p videos.
- On VBench-Long, LinGen gets a total score of 81.76%, close to Gen-3 (82.32%) and Kling (81.85%), and generates up to 1088 raw frames on one GPU.
- Human evaluation win rates vs. Gen-3, LumaLabs, and Kling are 50.5%, 52.1%, and 49.1%, respectively, and LinGen beats DiT-4B with a 75.6% win rate in overall quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace DiT self-attention with MATE | LinGen-4B vs DiT-4B · video generation human evaluation | win rate: 75.6% | Helped |
| replace DiT self-attention with MATE | LinGen-4B vs DiT-4B · 512p 68s video generation | FLOPs speed-up: 15× | Helped |
| replace DiT self-attention with MATE | LinGen-4B vs DiT-4B · 512p 68s video generation | latency speed-up: 11.5× | Helped |
| add the TESA TE-branch (ablation) | LinGen · 256p 17s video generation | video quality | Helped |
| add Rotary Major Scan (ablation) | LinGen · 256p 17s video generation | video quality | Helped |
| replace RMS with Zigzag scan (ablation) | LinGen · 256p 17s video generation | loss | No clear effect |
| replace RMS with Zigzag scan (ablation) | LinGen · 512p 17s video generation | latency: 102 -> 144 | Hurt |
| replace Mamba2 with Mamba (ablation) | LinGen · 512p 17s video generation | latency: 102 -> 127 | Hurt |
| replace Mamba2 with Mamba (ablation) | LinGen · 256p 17s video generation | video quality | No clear effect |
| add review tokens (ablation) | LinGen · 256p 17s video generation | video quality | Helped |
| add text-image pairs during text-to-video pre-training (ablation) | LinGen · 256p 17s video generation | video quality | Helped |
| fine-tune on a small set of high-quality videos (ablation) | LinGen · 256p 17s video generation | video quality | Helped |
Surprising findings
- LinGen adapts faster to longer token sequences than DiT, which the authors attribute to Mamba's length adaptation ability.
Limitations
- Training relies on a large proprietary licensed dataset (ShutterStock), limiting reproducibility.
- Quality still only comparable to, not clearly better than, commercial models in human evaluation.
Experimental setup
Datasets: ShutterStock text-image pairs ShutterStock text-video pairs RawFilm
Benchmarks: VBench-Long human A/B evaluation
Metrics: VBench quality score VBench semantic score VBench total score human win rate FLOPs latency
Baselines: DiT-4B Runway Gen-3 Kling 1.5 LumaLabs OpenSora V1.2 LinGen w/ Mamba
Builds on: Mamba2 Diffusion Transformer Matten Swin Transformer Movie Gen
Closest papers by meaning
- StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text (CVPR 2025 · similarity 0.90)
- Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis (CVPR 2024 · similarity 0.90)
- Grid Diffusion Models for Text-to-Video Generation (CVPR 2024 · similarity 0.90)
- ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way (CVPR 2025 · similarity 0.89)
- A Recipe for Scaling up Text-to-Video Generation with Text-free Videos (CVPR 2024 · similarity 0.89)
- OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.