Research map / Architectures and efficiency / RNNs and normalization
Convolutional Sequence to Sequence Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the GNMT word-piece model with fully convolutional ConvS2S | WMT'14 English-German translation · WMT'14 English-German | BLEU: 24.61 -> 25.16 | Helped |
| Replace the GNMT word-piece model with fully convolutional ConvS2S | WMT'14 English-French translation · WMT'14 English-French | BLEU: 38.95 -> 40.51 | Helped |
| Use an eight-model ConvS2S ensemble instead of GNMT + RL | WMT'14 English-German translation · WMT'14 English-German | BLEU: 26.30 -> 26.43 | Helped |
| Use an eight-model ConvS2S ensemble instead of GNMT + RL | WMT'14 English-French translation · WMT'14 English-French | BLEU: 41.16 -> 41.44 | Helped |
| Increase the ConvS2S ensemble from eight to ten models | ConvS2S on WMT'14 English-French · WMT'14 English-French | BLEU: 41.44 -> 41.62 | Helped |
| Add source position embeddings (ablation) | ConvS2S on WMT'14 English-German · WMT'14 English-German | BLEU: 21.3 -> 21.7 | Helped |
| Add target position embeddings (ablation) | ConvS2S on WMT'14 English-German · WMT'14 English-German | BLEU: 21.5 -> 21.7 | Helped |
| Add source and target position embeddings (ablation) | ConvS2S on WMT'14 English-German · WMT'14 English-German | BLEU: 21.2 -> 21.7 | Helped |
| Use attention in all five decoder layers instead of layers 1–4 (ablation) | ConvS2S on WMT'14 English-German · WMT'14 English-German | BLEU: 21.54 -> 21.63 | Helped |
| Use attention in decoder layers 4 and 5 as well as layers 1–3 (ablation) | ConvS2S on WMT'14 English-German · WMT'14 English-German | BLEU: 21.36 -> 21.63 | Helped |
| Use attention in decoder layers 3, 4, and 5 as well as layers 1 and 2 (ablation) | ConvS2S on WMT'14 English-German · WMT'14 English-German | BLEU: 21.47 -> 21.63 | Helped |
| Use attention in decoder layers 2–5 instead of only layer 1 (ablation) | ConvS2S on WMT'14 English-German · WMT'14 English-German | BLEU: 21.26 -> 21.63 | Helped |
Closest papers by meaning
- Parallelizing non-linear sequential models over the sequence length (ICLR 2024 · similarity 0.89)
- Kernel-Based Approaches for Sequence Modeling: Connections to Neural Methods (NeurIPS 2019 · similarity 0.89)
- Sequencer: Deep LSTM for Image Classification (NeurIPS 2022 · similarity 0.89)
- Sequence Accumulation and Beyond: Infinite Context Length on Single GPU and Large Clusters (AAAI 2025 · similarity 0.88)
- Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling (NeurIPS 2024 · similarity 0.88)
- Non-autoregressive Sequence-to-Sequence Vision-Language Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.