Research map / Architectures and efficiency / RNNs and normalization
Non-autoregressive Translation with Layer-Wise Prediction and Deep Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DSLP to Vanilla NAT | Vanilla NAT · WMT'14 EN-DE | BLEU: 21.18 -> 22.72 | Helped |
| Add DSLP to Vanilla NAT | Vanilla NAT · WMT'14 DE-EN | BLEU: 24.93 -> 25.83 | Helped |
| Add DSLP to Vanilla NAT | Vanilla NAT · WMT'16 EN-RO | BLEU: 29.15 -> 30.48 | Helped |
| Add DSLP to Vanilla NAT | Vanilla NAT · WMT'16 RO-EN | BLEU: 29.69 -> 31.46 | Helped |
| Add DSLP to CTC | CTC · WMT'14 EN-DE | BLEU: 25.72 -> 26.85 | Helped |
| Add DSLP to GLAT | GLAT · WMT'14 DE-EN | BLEU: 29.63 -> 29.90 | Helped |
| Add mixed training to DSLP | Vanilla NAT w/ DSLP · WMT'14 EN-DE | BLEU: 22.72 -> 24.17 | Helped |
| Add mixed training to DSLP | CTC w/ DSLP · WMT'16 RO-EN | BLEU: 34.24 -> 34.60 | Helped |
| Add layer-wise prediction (LP) alone (ablation) | Vanilla NAT · WMT'14 EN-DE; 6 layers | BLEU: 21.18 -> 21.22 (+0.04) | No clear effect |
| Add deep supervision (DS) alone (ablation) | Vanilla NAT · WMT'14 EN-DE; 6 layers | BLEU: 21.18 -> 21.84 (+0.66) | Helped |
| Combine layer-wise prediction and deep supervision (ablation) | Vanilla NAT · WMT'14 EN-DE; 6 layers | BLEU: 21.18 -> 22.72 (+1.54) | Helped |
| Add layer-wise prediction alone (ablation) | Vanilla NAT · WMT'14 EN-DE; 12 layers | BLEU: 22.22 -> 22.24 (+0.02) | No clear effect |
Closest papers by meaning
- Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation (ICLR 2021 · similarity 0.92)
- Improving Non-Autoregressive Translation Models Without Distillation (ICLR 2022 · similarity 0.91)
- Non-Autoregressive Neural Machine Translation (ICLR 2018 · similarity 0.91)
- RenewNAT: Renewing Potential Translation for Non-autoregressive Transformer (AAAI 2023 · similarity 0.89)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.87)
- AEQA-NAT : Adaptive End-to-end Quantization Alignment Training Framework for Non-autoregressive Machine Translation (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.