Research map / Video / Video generation
HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyperVAttention, a training-free clustering-based sparse attention for video diffusion transformers that combines 3D local-window clustering, hybrid anchor/subset clustering, and hardware-aware parallel cluster merging to reach up to 2.23x end-to-end speedup on HunyuanVideo.
Problem
Clustering-based sparse attention for video diffusion transformers is limited in practice by high clustering overhead (global k-means) and low CTA/thread utilization from irregular cluster-induced blocks.
Approach
Restrict centroid search to local 3D spatio-temporal windows, reuse cluster assignments across denoising steps by fully clustering only at anchor steps and refreshing a Lowe's-ratio-selected subset otherwise, and merge query clusters in parallel to minimize tiled CTA execution cost while filling idle tile capacity.
Key results
- On HunyuanVideo T2V, HVA reaches 33.97 PSNR and 0.046 LPIPS versus 32.66 PSNR and 0.056 LPIPS for SVG2.
- HVA-Flash achieves up to 2.23x end-to-end speedup on HunyuanVideo and 1.81x on Wan2.2 I2V.
- 3D local-window clustering reaches clustering error comparable to k-means at 50 iterations in 10 iterations (~40 ms vs >250 ms), a 6.29x latency reduction; the Triton kernel is up to 411x faster than the PyTorch version.
Surprising findings
- Cluster merging raised theoretical density (39.70% to 42.24%) yet lowered actual CTA density (50.42% to 46.88%), so higher density did not translate to higher cost.
Limitations
- Benefits are most pronounced for long videos where attention dominates runtime; gains may be smaller for images or very short videos.
- Cluster merging is a parallel heuristic that does not guarantee the global optimum of the partition problem.
- Evaluation limited to Wan2.2 and HunyuanVideo on 50 instances per task, with no error bars reported.
Experimental setup
Datasets: VBench prompts Penguin Benchmark prompts
Benchmarks: VBench Penguin Benchmark
Metrics: PSNR SSIM LPIPS VBench score LLM-Judge score end-to-end latency speedup CTA density clustering error
Baselines: SpargeAttn SVG SVG2 Flash k-means
Builds on: SVG2 FlashAttention SVG SpargeAttn Flash k-means
Closest papers by meaning
- Sparse Video-Gen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity (ICML 2025 · similarity 0.90)
- Faster Video Diffusion with Trainable Sparse Attention (NeurIPS 2025 · similarity 0.90)
- MonarchRT: Efficient Attention for Real-Time Video Generation (NeurIPS 2026 · similarity 0.89)
- Bidirectional Sparse Attention for Faster Video Diffusion Training (NeurIPS 2026 · similarity 0.89)
- iCATS: Fast Video Generation via Interaction-Aware Sparse Attention and Timestep-Adaptive Sparsity (NeurIPS 2026 · similarity 0.89)
- Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.