Research map / Video / Video generation

MonarchRT: Efficient Attention for Real-Time Video Generation

NeurIPS 2026 · Video › Video generation · approach: Efficient video diffusion models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Applies tiled Monarch matrix parameterizations, with block-aligned factorization and finetuning, to approximate 3D self-attention for real-time few-step autoregressive video diffusion, reaching ~95% sparsity with near-dense quality.

Problem

Quadratic 3D self-attention bottlenecks real-time video generation; sparse and low-rank approximations degrade quality in autoregressive, few-step diffusion settings.

Approach

Model 3D attention as positional (separable) plus sparse semantic components, align Monarch block sizes with the video (f,h,w) layout, generalize Monarch to tiled Monarch for controllable refinement, and finetune Monarch factors so a single refinement iteration suffices.

Key results

• On Self-Forcing, MonarchRT at 95% sparsity matches dense VBench total score (0.838 vs 0.836), while prior sparse methods degrade beyond ~85% sparsity.
• Training-free at 90% sparsity on Self-Forcing, MonarchRT reaches VBench total 0.839 vs exact top-k 0.799 and RadialAttention 0.816.
• Kernel speedups up to 11.8x over FA-2 on RTX 5090 and 5.6x over FA-3 on H100 at 720p; MonarchRT reaches 16 FPS on Self-Forcing on RTX 5090 versus 11 FPS for FA-2.

Surprising findings

• Exact oracle top-k attention at 10% density still produces poor-quality video, showing 3D attention is not truly sparse.
• Original Monarch approximation error does not reliably decrease with more computation, motivating tiled Monarch.

Limitations

• Monarch finetuning evaluated mainly on Wan-scale models; generalization to other architectures not shown.
• Some configurations (e.g., 720p Self-Forcing on RTX 5090) run out of memory or rely on theoretical 720p numbers.
• Parameter count and quality trade-offs depend on block alignment choices that require knowledge of video dimensions.

Experimental setup

Datasets: VBench MovieBench

Benchmarks: VBench MovieBench

Metrics: VBench quality score VBench semantic score VBench total score PSNR SSIM LPIPS latency (ms) FPS

Baselines: Dense Attention VSA Exact top-k SVG SVG2 RadialAttention FlashAttention-2 FlashAttention-3

Builds on: MonarchAttention Monarch parameterization Self-Forcing DMD

Closest papers by meaning

• Sparse Video-Gen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity (ICML 2025 · similarity 0.91)
• Faster Video Diffusion with Trainable Sparse Attention (NeurIPS 2025 · similarity 0.90)
• Training-Free Efficient Video Generation via Dynamic Token Carving (NeurIPS 2025 · similarity 0.90)
• MonarchAttention: Zero-Shot Conversion to Fast, Hardware-Aware Structured Attention (NeurIPS 2025 · similarity 0.90)
• PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers (NeurIPS 2026 · similarity 0.89)
• HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion (NeurIPS 2026 · similarity 0.89)