Research map / Video / Video generation

Bidirectional Sparse Attention for Faster Video Diffusion Training

NeurIPS 2026 · Video › Video generation · approach: Efficient video diffusion models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BSA, a trainable bidirectional sparse attention for video DiTs that prunes redundant queries within 3D blocks and dynamically selects KV blocks via statistical thresholds, reporting large training speedups on Wan2.1-1.3B with quality matching full attention.

Problem

Full attention in video diffusion transformers has quadratic cost in sequence length, making high-resolution long video training and inference very expensive; existing sparse methods mostly prune only KV pairs with fixed patterns.

Approach

Sparsify both queries and key-value pairs: within each 3D block, prune queries that are cosine-similar to a center token (with filling from the center output), and select KV blocks per query block using a dynamic statistical threshold on block-level attention scores, with Triton block-sparse kernels.

Key results

• At 153K tokens, BSA achieves 17.79x attention training speedup with about 5% of full-attention FLOPs while matching or exceeding full-attention VBench consistency scores on Wan2.1-1.3B.
• At 23K tokens, BSA reaches 12.85x speedup at 0.93 sparsity with validation loss 0.212 vs 0.213 for full attention.
• Training-free BSA achieves 4.6x and 6.2x speedups at 23K and 153K tokens on Wan2.1-1.3B with quality comparable to full attention.

Surprising findings

• Sparse training slightly outperforms full attention on several VBench consistency metrics and validation loss.
• Pruning 50% of queries alone yields lower validation loss than full attention.

Limitations

• Gains depend on the sparsity schedule and hyperparameters such as block size, retained ratio and window size.
• Evaluation is limited to VBench consistency metrics and a user study, with a single model family (Wan2.1).

Experimental setup

Datasets: Vchitect T2V DataVerse (300k videos) MovieGen-bench (user study prompts)

Benchmarks: VBench

Metrics: TextConsis BGConsis ImageQual SubConsist validation loss FLOPs speedup

Baselines: Full Attention MoBA VSA

Builds on: MoBA NSA FlashAttention Wan2.1

Closest papers by meaning

• Faster Video Diffusion with Trainable Sparse Attention (NeurIPS 2025 · similarity 0.94)
• DiTFastAttn: Attention Compression for Diffusion Transformer Models (NeurIPS 2024 · similarity 0.90)
• Sparse Video-Gen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity (ICML 2025 · similarity 0.90)
• PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers (NeurIPS 2026 · similarity 0.90)
• $\text{S}^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation (NeurIPS 2025 · similarity 0.90)
• Parameterized Stripe Attention for Efficient Video Generation (NeurIPS 2026 · similarity 0.90)