Research map / Label-efficient and robust learning / Recognition and long-tail
Mixed Cross Entropy Loss for Neural Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CE with mixed CE in teacher forcing | Transformer NMT · WMT'16 Ro-En test, single, beam 1 | BLEU: 30.63 -> 31.17 | Helped |
| replace CE with mixed CE in teacher forcing | Transformer NMT · WMT'16 Ru-En test, single, beam 1 | BLEU: 28.87 -> 29.59 | Helped |
| replace CE with mixed CE in teacher forcing | Transformer NMT · WMT'14 En-De test, single, beam 1 | BLEU: 26.23 -> 26.32 | Helped |
| replace CE with mixed CE in scheduled sampling | Transformer NMT · WMT'16 Ro-En test, single, beam 1 | BLEU: 30.71 -> 31.71 | Helped |
| replace CE with mixed CE in scheduled sampling | Transformer NMT · WMT'16 Ru-En test, single, beam 1 | BLEU: 29.28 -> 30.19 | Helped |
| replace CE with mixed CE in scheduled sampling | Transformer NMT · WMT'14 En-De test, single, beam 1 | BLEU: 26.36 -> 26.75 | Helped |
| replace CE with mixed CE | Transformer trained on WMT'14 En-De · WMT'19 En-De paraphrased test, sampling | BLEU: 8.89 -> 9.90 | Helped |
| replace the greedy auxiliary target with a randomly selected top-1 or top-2 token (ablation) | Mixed CE in scheduled sampling · WMT'16 Ro-En validation, scheduled sampling | BLEU: 33.64 -> 32.17 | Hurt |
| use a random vocabulary token as auxiliary target when greedy prediction differs from gold (ablation) | Mixed CE in scheduled sampling · WMT'16 Ro-En validation, scheduled sampling | BLEU: 33.64 -> 33.26 | Hurt |
| use the first-pass full probability distribution as auxiliary target (ablation) | Mixed CE in scheduled sampling · WMT'16 Ro-En validation, scheduled sampling | BLEU: 33.64 -> 32.03 | Hurt |
| add the second-pass greedy prediction as an auxiliary target (ablation) | Mixed CE in scheduled sampling · WMT'14 En-De validation, word oracle | BLEU: 26.88 -> 27.06 | Helped |
| vary the weight cap m in the mixed-loss schedule (ablation) | Mixed CE in teacher forcing and scheduled sampling · WMT'16 Ro-En validation | BLEU | Mixed |
Closest papers by meaning
- Scaling Laws for Neural Machine Translation (ICLR 2022 · similarity 0.91)
- Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation (ICML 2021 · similarity 0.88)
- Aligned Cross Entropy for Non-Autoregressive Machine Translation (ICML 2020 · similarity 0.88)
- Cut Your Losses in Large-Vocabulary Language Models (ICLR 2025 · similarity 0.88)
- What Scales in Cross-Entropy Scaling Law? (ICLR 2026 · similarity 0.86)
- Scaling Law with Learning Rate Annealing (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.