Research map / Video / Video generation
Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AVIS, which runs a streaming autoregressive video diffusion model (Self Forcing on Wan2.1-1.3B) as a zero-shot video inverse problem solver, initializing reverse diffusion from a CG-computed measurement-consistent estimate, plus AVIS Flash that applies measurement guidance only to the first chunk.
Problem
Zero-shot diffusion-based video inverse problem solvers restore videos holistically, causing high first-frame latency, and require repeated VAE passes for measurement consistency, causing low throughput.
Approach
Combine an autoregressive (KV-cached, chunk-wise) video diffusion backbone with a training-free measurement-consistent initialization (CG on the data-fidelity objective) that starts reverse diffusion at a small t0, and skip measurement guidance for all chunks after the first so that the restored prefix propagates through autoregressive context.
Key results
- AVIS reduces first-frame latency from 114 s (LVTINO) to 4 s and raises throughput from 0.71 to 1.18 FPS on 4x super-resolution on an RTX 4090.
- AVIS Flash reaches 5.91 FPS on a single RTX 4090 (over 8x LVTINO), and 10.2 FPS on an H100 with 1.85 s latency.
- AVIS achieves the best PSNR, SSIM, LPIPS and FVD on most tasks, e.g. inpainting PSNR 31.27 and FID 10.21 vs. LVTINO 26.36 and 35.75.
Surprising findings
- Removing measurement guidance for all chunks after the first still yields competitive restoration quality, with the restored prefix propagating via autoregressive context.
- Autoregressive propagation without initialization drifts away from the restoration target over time, so the initialization is needed to counter drift.
Limitations
- Still requires multiple VAE passes for the initial pixel-space guidance in AVIS.
- AVIS Flash shows a slight drop in some metrics versus AVIS and error drift over very long videos unless guidance is re-injected periodically.
- Baselines SVI and VDPS were excluded due to resolution or memory limits; several methods lacked public code.
Experimental setup
Datasets: Pexels
Benchmarks: Pexels 100-video set (super-resolution, inpainting, Gaussian deblur, temporal average, spatio-temporal average) GoPro240 Adobe240
Metrics: PSNR SSIM LPIPS FID FVD VBench subject consistency VBench background consistency VBench motion smoothness VBench aesthetic quality VBench imaging quality
Baselines: DiffIR2VR-Zero VISION-XL LVTINO
Builds on: Self Forcing CCDF Decomposed Diffusion Sampling flow matching Wan2.1
Closest papers by meaning
- Solving Video Inverse Problems Using Image Diffusion Models (ICLR 2025 · similarity 0.88)
- Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Models (NeurIPS 2024 · similarity 0.86)
- AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration (ICML 2025 · similarity 0.86)
- Lightning-Fast Image Inversion and Editing for Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.85)
- InvFusion: Bridging Supervised and Zero-shot Diffusion for Inverse Problems (NeurIPS 2025 · similarity 0.85)
- Diffusion Renderer: Neural Inverse and Forward Rendering with Video Diffusion Models (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.