Research map / Architectures and efficiency / RNNs and normalization
Sentence-Wise Smooth Regularization for Sequence to Sequence Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add sentence-wise smooth regularization to MLE | Transformer · WMT14 En→De, Transformer Base | BLEU: 27.30* -> 28.32 | Helped |
| Add sentence-wise smooth regularization to MLE | Transformer · WMT14 En→De, Transformer Big | BLEU: 28.40 -> 29.01 | Helped |
| Add sentence-wise smooth regularization to MLE | Transformer Big · WMT17 Zh→En | BLEU: 24.30 to 25.05 | Helped |
| Add sentence-wise smooth regularization to MLE | Transformer Small · IWSLT14 De→En | BLEU: 32.20 to 33.47 | Helped |
| Add sentence-wise smooth regularization to MLE | Transformer Small · Gigaword test | ROUGE-1: +1.0 | Helped |
| Add sentence-wise smooth regularization to MLE | Transformer Small · Gigaword test | ROUGE-2: +0.7 | Helped |
| Add sentence-wise smooth regularization to MLE | Transformer Small · Gigaword test | ROUGE-L: +0.9 | Helped |
| Focus updates on high-probability tokens using the curriculum baseline | Transformer · NMT tasks | BLEU | No clear effect |
| Add label smoothing without smooth regularization (ablation) | Transformer Small · IWSLT14 De→En | BLEU: 31.88 -> 32.20 | Helped |
| Add smooth regularization without label smoothing (ablation) | Transformer Small · IWSLT14 De→En | BLEU: 31.88 -> 33.04 | Helped |
| Add smooth regularization alongside label smoothing (ablation) | Transformer Small · IWSLT14 De→En | BLEU: 32.20 -> 33.47 | Helped |
| Set regularization weight λ too small or too large (ablation) | Smooth · IWSLT14 De→En | BLEU: 33.47; 33.19; 33.42; 33.39; 33.26 | Hurt |
Closest papers by meaning
- MLE-Guided Parameter Search for Task Loss Minimization in Neural Sequence Modeling (AAAI 2021 · similarity 0.89)
- Improving Sequence-to-Sequence Learning via Optimal Transport (ICLR 2019 · similarity 0.89)
- Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs (ICLR 2019 · similarity 0.87)
- Minimum Divergence vs. Maximum Margin: an Empirical Comparison on Seq2Seq Models (ICLR 2019 · similarity 0.87)
- Learning from others' mistakes: Finetuning machine translation models with span-level error annotations (ICML 2025 · similarity 0.84)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.