AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers

Yifan Zhou, Zeqi Xiao, Tianyi Wei, Shuai Yang, Xingang Pan

CVPR 2026 · Architectures and efficiency › Attention and transformers · approach: Linear attention approximations · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Hierarchical KV Enrichment to single-level Top-K attention (ablation)Top-K sparse attention (L=1) · FFHQ 128x128FID: 28.21 -> 26.09Improved
add KV Reweighting on top of KV Enrichment (ablation)Top-K sparse attention (L=1) · FFHQ 128x128FID: 26.09 -> 24.18Improved
add KV Reweighting at L=2 (ablation)LLSA · FFHQ 128x128FID: 25.31 -> 24.37Improved
extend single-level to two-level hierarchy (ablation)Top-K attention with KV Enrichment and Reweighting · FFHQ 128x128FID / throughput: FID 24.18 -> 24.37; throughput 302.92 -> 436.40Mixed
use block size 64 instead of 16 in Top-K attention (ablation)Top-K attention (L=1) · FFHQ 128x128FID: 28.21 -> 31.33 (K=8 vs K=2); 25.88 -> 26.52 (K=32 vs K=8)Hurt
increase K of single-level baseline from 8 to 32 (ablation)Top-K attention (L=1) · FFHQ 128x128FID: 28.21 -> 25.88; throughput 483.91 -> 357.95Improved
use LLSA with K=8 instead of baseline Top-K with K=32 (ablation)Top-K sparse attention · FFHQ 128x128FID: 25.88 -> 24.37Improved
replace VSA/SLA with LLSADiT-S on FFHQ 256x256 · FFHQ 256x256FID: 39.98 -> 39.29 vs SLA; 40.69 -> 39.29 vs VSAImproved
replace full attention with LLSApixel DiT-S · FFHQ 256x256FID: 38.77 -> 39.29; throughput 61.64 -> 375.34Mixed
replace VSA/SLA with LLSA in PixelFlow at highest resolution stagePixelFlow · ImageNet-256FID: 22.58 -> 20.41 vs SLAImproved
replace mask-based key-value backward with sparse index transpose kernelLLSA · Efficiency benchmarkbackward throughputImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy