Research map / Video / Video generation
AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AR-Diffusion, an asynchronous video diffusion model with a non-decreasing per-frame timestep constraint, a balanced FoPP training timestep scheduler, and an adaptive-difference (AD) inference scheduler, built on a causal video autoencoder (AR-VAE) and DiT.
Problem
Video generation methods are either auto-regressive (asynchronous, but suffer train-inference mismatch and error accumulation) or synchronous diffusion (same noise on all frames, limited to fixed-length sequences); asynchronous diffusion with independent per-frame noise has a huge search space and u
Approach
Apply diffusion corruption to frames during both training and inference with a non-decreasing timestep constraint (earlier frames cleaner), temporal causal attention, a dynamic-programming-based sampler (FoPP) to balance timestep composition sampling, and an inference scheduler with adjustable timestep difference s that interpolates between synchronous (s=0) and auto-regressive (s large) generatio
Key results
- AR-Diffusion achieves FVD16 of 186.6 on UCF-101 (s=15) and 66.3 on Taichi-HD (s=10), with FVD16 of 40.8 on Sky-Timelapse, lower than all compared methods.
- Reports 60.1% FVD improvement over the prior asynchronous diffusion model FVDM on UCF-101.
- Ablation on Sky-Timelapse: removing the non-decreasing constraint raises FVD from 62.8 to 272.5; removing FoPP raises it to 101.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add FoPP Timestep Scheduler (ablation) | AR-Diffusion · Sky-Timelapse | FVD: 101.0 -> 62.8 | Helped |
| use the Improved VAE (ablation) | AR-Diffusion · Sky-Timelapse | FVD: 148.3 -> 62.8 | Helped |
| add Temporal Causal Attention (ablation) | AR-Diffusion · Sky-Timelapse | FVD: 209.8 -> 62.8 | Helped |
| use x0 prediction loss instead of ϵ prediction (ablation) | AR-Diffusion · Sky-Timelapse | FVD: 257.6 -> 62.8 | Helped |
| add the non-decreasing timestep constraint (ablation) | AR-Diffusion · Sky-Timelapse | FVD: 272.5 -> 62.8 | Helped |
| set timestep difference s=5 instead of s=0 | AR-Diffusion, 16-frame generation · Taichi-HD | FVD: 80.9 -> 66.3 | Helped |
| set timestep difference s=15 instead of s=0 | AR-Diffusion, 16-frame generation · UCF-101 | FVD: 194.4 -> 186.6 | Helped |
| increase timestep difference beyond s=15 | AR-Diffusion, 16-frame generation · multiple datasets | FVD | Hurt |
| use s=5 for 128-frame generation | AR-Diffusion · multiple datasets | FVD | Helped |
Surprising findings
- The best timestep difference s is dataset-dependent: s=0 (synchronous) is best on FaceForensics and Sky-Timelapse, while intermediate values work best on Taichi-HD and UCF-101.
Limitations
- Gains on FVD128 for some datasets are mixed; the best s varies by dataset and must be tuned.
- Inference cost increases with larger timestep difference s.
Experimental setup
Datasets: FaceForensics Sky-Timelapse Taichi-HD UCF-101
Benchmarks: FaceForensics Sky-Timelapse Taichi-HD UCF-101
Metrics: FVD16 FVD128 FID-img FID-vid
Baselines: MoCoGAN DIGAN StyleGAN-V MoStGAN-V VideoGPT TATS Latte PVDM LVDM VIDM FVDM Diffusion Forcing
Builds on: DiT TiTok Diffusion Forcing DDIM
Closest papers by meaning
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.92)
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (NeurIPS 2025 · similarity 0.91)
- Video-Mirai: Autoregressive Video Diffusion Models Need Foresight (NeurIPS 2026 · similarity 0.90)
- ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generation (ICLR 2025 · similarity 0.90)
- Generative Pre-trained Autoregressive Diffusion Transformer (NeurIPS 2025 · similarity 0.90)
- AdaState: Self-Evolving Anchors for Streaming Video Generation (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.