Research map / Architectures and efficiency / Attention and transformers
Forge: Compiling a Unified Abstraction into Scalable Kernels for Linear Attention
Haojie Duanmu, Size Zheng, Ningxin Zheng, Jianqiao Lu, Xuegui Zheng, Xingcheng Zhang, Li-Wen Chang, Xin Liu, Dahua Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AOT compilation with static dispatcher (ablation) | FlexLA Scalar GLA · seq len 1024 | dispatch overhead / end-to-end latency: reduces overhead by 46%, 1.6x end-to-end speedup | Improved |
| add AOT compilation with static dispatcher (ablation) | FlexLA Scalar GLA · seq len 8192 | dispatch overhead: 101 us -> 1 us | Improved |
| add tile-level compute-communication fusion instead of serial communication (ablation) | FlexLA state propagation · 8xH800 67MB state | communication time: 873 us -> 560 us | Improved |
| use host-managed pipelined NCCL overlap instead of serial (ablation) | state propagation · 8xH800 67MB state | communication time: 873 -> 902 | Hurt |
| replace PyTorch eager with FlexLA compiled kernel | scalar GLA · single GPU | latency: 34.6 s -> 9.2 ms | Improved |
| scale FlexLA from one GPU to 4 GPUs | scalar GLA · Figure 1 | latency: 9.2 ms -> 2.7 ms | Improved |
| replace FLA kernels with FlexLA generated kernels | HGRN · single H100 | speedup: 1.64-2.02x | Improved |
| replace FLA kernels with FlexLA generated kernels | scalar GLA, vector GLA, vanilla LA · single H100 | speedup: 1.1-2.0x | Improved |
| replace FLA kernels with FlexLA generated kernels | GDN · single H100 long sequences | latency | No clear effect |
| replace LASP2 all-gather with FlexLA fused communication | GDN · weak scaling to 128 GPUs | latency: LASP2 49.2ms on 4 GPUs to 345ms on 128 GPUs; FlexLA flat | Improved |
| replace LASP-2 with FlexLA | scalar GLA · H100, 4 GPUs, 16k tokens/GPU | latency (ms): 1.13 -> 0.96 | Improved |
| replace Torch-Compile with domain-specific compiler | linear attention · single H100 | latency | Improved |
Closest papers by meaning
- The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry (ICLR 2024 · similarity 0.86)
- STILL: Selecting Tokens for Intra-Layer Hybrid Attention to Linearize LLMs (NeurIPS 2026 · similarity 0.86)
- Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels (NeurIPS 2025 · similarity 0.86)
- Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention (ICML 2024 · similarity 0.85)
- Sparse Attention as Compact Kernel Regression (NeurIPS 2026 · similarity 0.85)
- Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
