Research map / Video / Video generation
AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AsymRnR, a training-free, model-agnostic method that asymmetrically reduces query and key/value tokens in video DiT attention, with scheduling and a matching cache, to accelerate video generation.
Problem
Video Diffusion Transformers are slow due to quadratic attention cost over long video token sequences; existing token-merging and caching accelerators either need fine-tuning, are architecture-specific, or cause distortions when applied uniformly to video DiTs.
Approach
Reduce Q and K&V independently (asymmetric reduction) before attention and restore only the Q sequence afterward, allocate reduction rates per feature, block and timestep via a similarity-based schedule estimated in advance, use Euclidean matching instead of cosine and discard instead of merge, and cache matching results across denoising steps.
Key results
- On CogVideoX-2B, AsymRnR reaches VBench 0.7917 at 1.13x speedup versus baseline 0.8008 and ToMe 0.7825 at 1.11x; the fast variant reaches 1.17x.
- On HunyuanVideo, AsymRnR gives 1.24x speedup with VBench 0.8240 vs baseline 0.8157; the fast variant gives 1.30x.
- Stacking AsymRnR on PAB for HunyuanVideo gives 1.71x overall speedup with VBench 0.8140 vs 0.8157.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AsymRnR instead of ToMe | CogVideoX-2B · VBench | VBench: 0.7825 -> 0.7917 | Helped |
| use AsymRnR instead of the baseline | HunyuanVideo · VBench | VBench: 0.8157 -> 0.8240 | Helped |
| apply AsymRnR to FastVideo-Hunyuan | FastVideo-Hunyuan · sampling latency | speedup: +20% (1.20 ×) | Helped |
| add AsymRnR on top of PAB | HunyuanVideo with PAB · HunyuanVideo | speedup: 1.28× -> 1.71× | Helped |
| schedule reductions selectively across blocks and timesteps (ablation) | AsymRnR on CogVideoX-2B · VBench | VBench: 0.7641 -> 0.7893 | Helped |
| add scheduled Q reduction alongside scheduled V reduction (ablation) | AsymRnR on CogVideoX-2B · VBench | VBench: 0.7787 -> 0.7917 | Helped |
| increase matching cache steps from s=1 to s=5 (ablation) | V reduction in AsymRnR on CogVideoX-2B · CogVideoX-2B | latency: 134.68 -> 118.16 | Helped |
| increase matching cache steps from s=1 to s=5 (ablation) | V reduction in AsymRnR on CogVideoX-2B · VBench | VBench: 0.7859 -> 0.7796 | Hurt |
| use Euclidean rather than cosine similarity for matching (ablation) | AsymRnR on CogVideoX-2B · VBench | VBench: 0.7716 -> 0.7796 | Helped |
| discard redundant tokens rather than merge them by mean (ablation) | AsymRnR on CogVideoX-2B · VBench | VBench: 0.7630 -> 0.7796 | Helped |
| use mean reduction instead of direct discarding (ablation) | AsymRnR on CogVideoX-2B · latency | latency: 118.16 -> 119.90 | Hurt |
| use destination stride (3,2,2) rather than (2,2,2) (ablation) | AsymRnR on CogVideoX-2B · VBench | VBench: 0.7796 -> 0.7417 | Hurt |
Surprising findings
- On Mochi-1 and HunyuanVideo, AsymRnR slightly exceeds baseline VBench scores while reducing compute.
- Cosine similarity, the standard ToMe choice, performs worse than negative Euclidean distance for matching in this setting.
Limitations
- Visual discrepancies remain in some generated outputs despite semantic consistency.
- Similarity thresholds and reduction rates must be tuned per baseline model.
- Minor acceleration for image-based DiTs due to short sequences.
Experimental setup
Datasets: VBench prompts
Benchmarks: VBench
Metrics: VBench score LPIPS FLOPs latency speedup
Baselines: ToMe ToMe-fast PAB
Builds on: Token Merging (ToMe) Bipartite Soft Matching PAB FastVideo
Closest papers by meaning
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.90)
- Towards Precise Scaling Laws for Video Diffusion Transformers (CVPR 2025 · similarity 0.90)
- ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler (ICLR 2025 · similarity 0.90)
- SparseDiT: Token Sparsification for Efficient Diffusion Transformer (NeurIPS 2025 · similarity 0.89)
- HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration (ICML 2025 · similarity 0.89)
- Bidirectional Sparse Attention for Faster Video Diffusion Training (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.