Research map / Architectures and efficiency / Attention and transformers
Online and Linear-Time Attention by Enforcing Monotonic Alignments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use hard monotonic decoding instead of offline softmax attention | Sequence-to-sequence speech recognition model · WSJ eval92 | WER: 16.0% -> 17.4% | Hurt |
| Use hard monotonic decoding instead of soft monotonic attention | TIMIT speech recognition model · TIMIT test | PER: 20.1% -> 20.4% | Hurt |
| Use hard monotonic decoding instead of softmax attention | Gigaword summarization model · Gigaword test | ROUGE R-1: 38.22 -> 37.14 | Hurt |
| Use hard monotonic decoding instead of soft monotonic attention | Gigaword summarization model · Gigaword test | ROUGE R-2: 18.57 -> 18.00 | Hurt |
| Use hard monotonic decoding instead of softmax attention | English-Vietnamese translation model · IWSLT 2015 English-Vietnamese TED tst2013 | BLEU: 23.3 -> 22.6 | Hurt |
| Replace additive energy with dot-product energy (ablation) | Hard monotonic English-Vietnamese model · IWSLT 2015 English-Vietnamese TED tst2013 | BLEU: 22.6 -> 23.0 | Helped |
| Use hard monotonic attention rather than standard softmax attention | Attention mechanism · Synthetic attention speed benchmark | speedup: 4-40× faster | Helped |
Closest papers by meaning
- Monotonic Chunkwise Attention (ICLR 2018 · similarity 0.91)
- Posterior Attention Models for Sequence to Sequence Learning (ICLR 2019 · similarity 0.89)
- Monotonic Multihead Attention (ICLR 2020 · similarity 0.88)
- Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study (ICLR 2025 · similarity 0.86)
- Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks (NeurIPS 2025 · similarity 0.85)
- RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.