Research map / Video / Video generation
InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InfinityStar, a purely discrete spacetime autoregressive framework (built on Infinity/VAR next-scale prediction) that unifies text-to-image, text-to-video, image-to-video and long interactive video generation, reaching VBench 83.74 with ~32x speedup over Wan 2.1.
Problem
Autoregressive visual generators lag diffusion models in video fidelity and suffer high latency; discrete video tokenizers are hard to train and existing AR video models need thousands of inference steps.
Approach
Model a video as an image pyramid plus per-clip spacetime pyramids that grow only spatially, initialize the discrete tokenizer from a continuous video VAE (knowledge inheritance), regularize with stochastic quantizer depth, repeat early semantic scales, and attend sparsely to only the previous clip's last scale.
Key results
- InfinityStar scores 83.74 on VBench, above HunyuanVideo (83.24) and the best open autoregressive baselines.
- InfinityStar-T2I reaches GenEval overall 0.79 (with prompt rewriter) and DPG overall 86.55.
- Generates a 5s 720p video in 58 s on one GPU, a 32x speedup over Wan 2.1 (1864 s), and about 6x over Nova.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Semantic Scale Repetition (ablation) | InfinityStar · VBench (1M 192p ablation) | Total score: 75.72 to 81.28 | Helped |
| use Spacetime Pyramid instead of Pseudo-Spacetime Pyramid (ablation) | InfinityStar · VBench (1M 192p ablation) | Total score: 80.30 to 81.28 | Helped |
| add Stochastic Quantizer Depth (ablation) | InfinityStar tokenizer · VBench (1M 192p ablation) | Total score: 81.07 to 81.28 | Helped |
| use Spacetime Sparse Attention instead of Full Attention (ablation) | InfinityStar · VBench (1M 192p ablation) | Total score: 80.77 to 81.28 | Helped |
| attend to the former clip's 3rd largest scale instead of its largest scale (ablation) | InfinityStar attention · VBench (1M 192p ablation) | Total score: 81.28 to 80.86 | Hurt |
| attend to the former clip's 6th largest scale instead of its largest scale (ablation) | InfinityStar attention · VBench (1M 192p ablation) | Total score: 81.28 to 80.75 | Hurt |
| inherit weights from a Continuous Video VAE instead of using Image VAE weights (ablation) | InfinityStar tokenizer · Internal high-motion video benchmark (480p 81 frames) | PSNR: 29.10 to 33.37 | Helped |
| inherit weights from a Continuous Video VAE instead of training with no pretrained weights (ablation) | InfinityStar tokenizer · Internal high-motion video benchmark (480p 81 frames) | PSNR: 30.04 to 33.37 | Helped |
| use Spacetime Sparse Attention instead of Full Attention (ablation) | InfinityStar · 480p 161 frames | Latency: OOM to 44.7s / 63 GB | Helped |
| use InfinityStar instead of Wan 2.1 | 720p, 5s, 81 frames video generation · Inference latency | Speedup: 32 | Helped |
Surprising findings
- Spacetime sparse attention (attending only the previous clip's last scale) scores higher on VBench (81.28) than full attention (80.77) while being faster and cheaper.
- Conditioning on the previous clip's last scale and a semantic downsampled condition is needed; attending to smaller scales degrades VBench to 80.75.
Limitations
- Trade-off between image quality and motion fidelity in high-motion scenes; fine details can be compromised.
- Cumulative errors degrade quality over many interaction rounds in long interactive generation.
- Model size and training scale are smaller than leading diffusion models; inference pipeline not fully optimized.
Experimental setup
Datasets: Panda-70M Mira internal text-image and text-video corpora synthetic interaction data
Benchmarks: VBench GenEval DPG-Bench
Metrics: VBench overall score GenEval overall DPG overall PSNR SSIM LPIPS inference latency
Baselines: HunyuanVideo Wan 2.1 Nova Emu3 CogVideoX-5B Infinity Goku OpenSora V1.2
Builds on: Infinity VAR Switti Wan 2.1 VAE FramePack
Closest papers by meaning
- Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis (CVPR 2025 · similarity 0.91)
- VideoMAR: Autoregressive Video Generation with Continuous Tokens (NeurIPS 2025 · similarity 0.90)
- Autoregressive Video Generation without Vector Quantization (ICLR 2025 · similarity 0.89)
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NeurIPS 2024 · similarity 0.88)
- StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generation (CVPR 2025 · similarity 0.88)
- Parallelized Autoregressive Visual Generation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.