Research map / Architectures and efficiency / Attention and transformers
Word Attention for Sequence to Sequence Text Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add word attention over source word embeddings (ablation) | RNNsearch · Gigaword summarization | ROUGE-1 F1: 33.67 -> 35.64 | Helped |
| add contextual gates on top of word attention (ablation) | RNNsearch +Word Attention · Gigaword summarization | ROUGE-1 F1: 35.64 -> 35.93 | Helped |
| add word attention (ablation) | RNNsearch (1-layer) · IWSLT14 De-En word | BLEU: 26.98 -> 27.39 | Helped |
| add contextual gates on top of word attention (ablation) | RNNsearch (1-layer) · IWSLT14 De-En word | BLEU: 27.39 -> 27.72 | Helped |
| add word attention (ablation) | RNNsearch (1-layer) · IWSLT14 De-En BPE | BLEU: 27.83 -> 28.49 | Helped |
| add contextual gates on top of word attention (ablation) | RNNsearch (1-layer) · IWSLT14 De-En BPE | BLEU: 28.49 -> 28.70 | Helped |
| add word attention (ablation) | 2-2 RNNsearch · IWSLT14 De-En word | BLEU: 29.01 -> 29.68 | Helped |
| add contextual gates on top of word attention (ablation) | 2-2 RNNsearch · IWSLT14 De-En BPE | BLEU: 31.71 -> 31.90 | Helped |
| add word attention with gates | 4-4 RNNsearch · WMT14 En-Fr | BLEU: 38.50 -> 39.10 | Helped |
| add word attention with gates | RNNsearch · Gigaword summarization | ROUGE-L F1: 31.67 -> 33.41 | Helped |
Closest papers by meaning
- Posterior Attention Models for Sequence to Sequence Learning (ICLR 2019 · similarity 0.89)
- RNN-Based Sequence-Preserved Attention for Dependency Parsing (AAAI 2018 · similarity 0.88)
- Focused Hierarchical RNNs for Conditional Sequence Processing (ICML 2018 · similarity 0.88)
- Online and Linear-Time Attention by Enforcing Monotonic Alignments (ICML 2017 · similarity 0.88)
- Not All Attention Is Needed: Gated Attention Network for Sequence Data (AAAI 2020 · similarity 0.88)
- RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.