Research map / Video / Video generation
Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Turns a pre-trained 8B latent video diffusion transformer into a one-step, per-latent-frame autoregressive streaming generator via diffusion adaptation, consistency distillation, and student-forcing adversarial post-training with long-video training.
Problem
Large video diffusion models are too computationally expensive for real-time, interactive streaming generation, and existing diffusion-forcing approaches need several denoising steps and suffer error accumulation over long videos.
Approach
Use adversarial post-training to make a block-causal DiT generate each latent frame in one forward pass with KV cache, recycling generated frames as inputs; student-forcing adversarial training (no paired ground truth) reduces error accumulation and allows training on long-duration segments via per-segment discrimination.
Key results
- 8B AAPT model generates 24 fps streaming video at 736x416 on a single H100 (0.16s latency) and 1280x720 at 24.2 fps on 8 H100s, versus CausVid 9.4 fps at 640x352 with 1.30s latency.
- On 1440-frame VBench-I2V, AAPT reaches temporal quality 89.79 and frame quality 62.16, the best among compared methods including SkyReel-V2 and MAGI-1.
- On camera-conditioned world exploration, AAPT achieves FVD 61.33 vs. 73.11 for CameraCtrl2, the best in three of six metrics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace diffusion baseline with AAPT | Video generator · VBench-I2V, 120 frames | Frame Quality: 66.08 -> 66.58 | Helped |
| replace diffusion baseline with AAPT | Video generator · VBench-I2V, 120 frames | Temporal Quality: 90.40 -> 89.51 | Hurt |
| replace diffusion baseline with AAPT | Video generator · VBench-I2V, 1440 frames | Temporal Quality: 86.65 -> 89.79 | Helped |
| replace diffusion baseline with AAPT | Video generator · VBench-I2V, 1440 frames | Frame Quality: 60.49 -> 62.16 | Helped |
| train for 60s instead of 10s (ablation) | AAPT · One-minute VBench-I2V generation | Temporal Quality: 85.86 -> 89.79 | Helped |
| train for 60s instead of 10s (ablation) | AAPT · One-minute VBench-I2V generation | Frame Quality: 57.92 -> 62.16 | Helped |
| use student-forcing instead of teacher-forcing adversarial training (ablation) | AAPT · Autoregressive video generation | error accumulation | Helped |
| add result recycling as input to subsequent autoregressive steps (ablation) | AAPT · Video generation with large motion | motion coherence | Helped |
| add long-video training (ablation) | AAPT · One-minute video generation | video quality | Helped |
| use a lightweight VAE decoder (ablation) | AAPT video decoder · VAE decoding | speed: nearly 3 times speed-up | Helped |
| use AAPT instead of diffusion baseline | Video generator · VBench-I2V, 1440 frames | I2V Subject: 95.01 -> 96.11 | Helped |
| use AAPT instead of diffusion baseline | Video generator · VBench-I2V, 120 frames | I2V Background: 99.14 -> 99.36 | Helped |
Surprising findings
- Teacher-forcing adversarial training fails to generate proper videos at inference, with content drifting within a few frames, while student-forcing is critical.
- Models trained only on 10-second clips do not generalize to one-minute videos, whereas long-video segment-based adversarial training substantially improves long-duration quality.
Limitations
- Difficulty maintaining subject and scene consistency, caused by both the sliding-window generator and segment-based discriminator.
- One-step generation can introduce defects that persist over time; zero-shot five-minute generation produces artifacts.
- Long-video adversarial training is slow.
Experimental setup
Datasets: proprietary video training data (average shot ~8s) VBench-I2V prompts pose-conditioned human video data camera-conditioned world exploration data
Benchmarks: VBench-I2V pose-conditioned human video generation test set camera-conditioned world exploration protocol (CameraCtrl II)
Metrics: VBench temporal quality VBench frame quality I2V subject consistency I2V background consistency AKD IQA ASE FID FVD Mov
Baselines: CausVid Wan2.1 HunyuanVideo MAGI-1 SkyReel-V2 DisCo AnimateAnyone MimicMotion CyberHost OmniHuman-1 MotionCtrl CameraCtrl
Builds on: APT CausVid diffusion forcing R3GAN consistency distillation
Closest papers by meaning
- Diffusion Adversarial Post-Training for One-Step Video Generation (ICML 2025 · similarity 0.91)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.90)
- Generative Pre-trained Autoregressive Diffusion Transformer (NeurIPS 2025 · similarity 0.88)
- AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion (CVPR 2025 · similarity 0.88)
- OSV: One Step is Enough for High-Quality Image to Video Generation (CVPR 2025 · similarity 0.88)
- Autoregressive Video Generation without Vector Quantization (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.