Research map / Video / Video generation
Video-Mirai: Autoregressive Video Diffusion Models Need Foresight
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Video-Mirai adds a training-only foresight loss that aligns a causal video diffusion generator's mid-depth hidden states to a frozen bidirectional encoder's features of its own future rollout, improving Causal-Forcing on VBench without changing inference.
Problem
Autoregressive streaming video diffusion is trained with present-segment losses that do not constrain whether causal hidden states retain identity, layout, and motion needed for future segments, causing long-horizon inconsistency.
Approach
Use future frames only as stopped-gradient supervision: a frozen Wan-14B encoder reads the completed causal rollout non-causally, and a discarded 3-block DiT predictor aligns the causal generator's layer-15 hidden states to the fused future encoder features (cosine loss), so the deployed model stays strictly causal.
Key results
- On 5-second VBench, Video-Mirai raises chunk-wise Causal-Forcing Total Score from 83.82 to 84.62 (Quality 84.54 to 85.38).
- On 30-second rollouts, subject consistency improves from 84.93 to 88.47 and background consistency from 90.22 to 91.94 over chunk-wise Causal-Forcing.
- Paired prompt-level bootstrap shows gains on Quality, Semantic, and Total scores with 95% CIs excluding zero; per-cycle training overhead measured at 13.7% with no inference overhead.
Surprising findings
- Fusing future encoder targets across offsets outperforms averaging per-offset losses, and MSE or SIGReg regularization degrades results versus plain cosine loss.
- Applying foresight at Stage 2 (ODE) and Stage 3 (DMD) together collapses to a smaller gain (Total 84.05) than either stage alone.
Limitations
- Foresight window is small (at most one chunk ahead, about 3 latent frames); extending to two segments degrades Quality.
- Gains at 5 seconds are modest (Total +0.8), and no multi-seed training variance is reported due to cost.
- Requires a large bidirectional Wan-14B encoder during training, adding roughly 14% training compute per cycle.
Experimental setup
Datasets: VidProM MovieGen prompts
Benchmarks: VBench MovieGen prompts 30-second rollouts
Metrics: Quality Score Semantic Score Total Score subject consistency background consistency overall consistency PSNR MSE LPIPS throughput (FPS)
Baselines: Causal-Forcing Self-Forcing CausVid Wan2.1 LTX-Video SkyReels-V2 MAGI-1 Pyramid Flow NOVA Rolling-Forcing
Builds on: Causal-Forcing Self-Forcing Distribution Matching Distillation REPA V-JEPA Mirai
Closest papers by meaning
- RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO (NeurIPS 2026 · similarity 0.91)
- AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion (CVPR 2025 · similarity 0.90)
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (NeurIPS 2025 · similarity 0.89)
- Generative Pre-trained Autoregressive Diffusion Transformer (NeurIPS 2025 · similarity 0.89)
- Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing (ICML 2025 · similarity 0.88)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.