Research map / Video / Video generation
Taming Teacher Forcing for Masked Autoregressive Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MAGI, a hybrid masked-intra-frame / causal-inter-frame video generator trained with Complete Teacher Forcing, dynamic interval training, and dynamic noise injection.
Problem
Autoregressive video generation either uses patch-level raster order (slow, ignores frame causality) or masked frame prediction that suffers a training-inference gap from masked teacher forcing, which hurts motion coherence and exposure bias.
Approach
Condition each masked frame on complete (unmasked) observation frames during training via prepended observation tokens and a special temporal attention mask, so training matches the KV-cache causal inference; add dynamic interval embeddings and noise-level embeddings to counter exposure bias.
Key results
- CTF outperforms MTF by about 23% in FVD on first-frame conditioned UCF-101 video prediction.
- MAGI achieves FVD64,50K of 11.5 on Kinetics-600 video prediction, best among autoregressive methods and versus Omni's 32.9.
- On unconditional UCF-101 generation, MAGI reaches FVD256,2048 of 297.8 with Cosmos VAE and 420.6 with SD1.4 VAE, beating Latte (477.9) with the same VAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Complete Teacher Forcing instead of Masked Teacher Forcing (ablation) | MAGI · UCF-101 first-frame conditioned video prediction | FVD: +23% | Helped |
| use Complete Teacher Forcing instead of Masked Teacher Forcing (ablation) | MAGI · UCF-101 first-frame conditioned video prediction | FID | Hurt |
| add dynamic interval training (ablation) | MAGI with Complete Teacher Forcing · UCF-101 first-frame conditioned video prediction | FVD | Helped |
| add dynamic noise injection (ablation) | MAGI with Complete Teacher Forcing · UCF-101 first-frame conditioned video prediction | FVD | Helped |
| add both dynamic interval training and dynamic noise injection (ablation) | MAGI with Complete Teacher Forcing · UCF-101 first-frame conditioned video prediction | FVD | Helped |
| add dynamic interval training (ablation) | MAGI with Complete Teacher Forcing · UCF-101 first-frame conditioned video prediction | FID | Helped |
| add dynamic noise injection (ablation) | MAGI with Complete Teacher Forcing · UCF-101 first-frame conditioned video prediction | FID | Helped |
Surprising findings
- Masked teacher forcing gives better per-frame FID but substantially worse FVD than CTF, i.e. better single-frame quality does not translate to better video coherence.
Limitations
- Performance degrades on non-periodic motions such as diving in long-horizon prediction.
- Evaluation on UCF-101 is limited in complexity and at 64x64 or 256x256 resolution.
Experimental setup
Datasets: Kinetics-600 UCF-101
Benchmarks: Kinetics-600 UCF-101
Metrics: FVD FID
Baselines: Video Diffusion RIN MAGVIT MAGVIT-v2 ViVQVAE Phenaki Omni MoCoGAN MoCoGAN-HD DIGAN StyleGAN-V PVDM
Builds on: MAR MaskGIT Genie GameNGen Omni
Closest papers by meaning
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (NeurIPS 2025 · similarity 0.89)
- Autoregressive Video Generation without Vector Quantization (ICLR 2025 · similarity 0.88)
- VideoMAR: Autoregressive Video Generation with Continuous Tokens (NeurIPS 2025 · similarity 0.88)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.88)
- AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion (CVPR 2025 · similarity 0.87)
- Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.