Research map / Architectures and efficiency / RNNs and normalization
A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond
Yisheng Xiao, Lijun Wu, Junliang Guo, Juntao Li, Min Zhang, Tao Qin, Tie‐Yan Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add correction and reveal-position to CMLM | CMLM · WMT14 EN-DE | BLEU: 27.03* -> 28.37* | Improved |
| use Viterbi decoding instead of default decoding | DA-Transformer · WMT14 EN-DE | BLEU: 27.49 -> 26.89 | Hurt |
| add fuzzy alignments to DA-Transformer | DA-Transformer · WMT14 EN-DE | BLEU: 27.49 -> 27.53 | Improved |
| add glancing training to NAT | NAT · WMT14 EN-DE | BLEU: 17.69 -> 25.21 | Improved |
| add iterative refinement to NAT | NAT · WMT14 EN-DE | BLEU: 17.69 -> 21.61 | Improved |
| increase decoding iterations (iterative vs one-step) | NAT · WMT14 EN-DE | decoding speedup: 15.6x -> 1.5x | Hurt |
| add NAT drafting with AT verification | semi-autoregressive decoding · WMT14 EN-DE | BLEU: 26.90 -> 28.89* | Improved |
| add sequence-level knowledge distillation (GLAT vs CMLM-like KD models) | one-step NAT · WMT14 EN-DE | BLEU: 25.21 -> 27.03* | Mixed |
| add CTC loss with order-agnostic loss | order-agnostic cross-entropy · WMT14 EN-DE | BLEU: 26.10* -> 27.41 | Improved |
| add n-gram level order-agnostic loss | order-agnostic cross-entropy · WMT14 EN-DE | BLEU: 26.10* -> 26.50* | Improved |
| add pre-trained XLM-R with decorator | NAT · WMT14 EN-DE | BLEU: 27.49 -> 29.80 | Improved |
| add deep supervision layer-wise predictions | one-step NAT · WMT16 EN-RO | BLEU: 31.19 -> 34.17 | Improved |
Closest papers by meaning
- Non-Autoregressive Neural Machine Translation (ICLR 2018 · similarity 0.93)
- Improving Non-Autoregressive Translation Models Without Distillation (ICLR 2022 · similarity 0.92)
- Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input (AAAI 2019 · similarity 0.91)
- Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation (AAAI 2020 · similarity 0.91)
- An EM Approach to Non-autoregressive Conditional Sequence Generation (ICML 2020 · similarity 0.91)
- AEQA-NAT : Adaptive End-to-end Quantization Alignment Training Framework for Non-autoregressive Machine Translation (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
