Research map / Label-efficient and robust learning / Recognition and long-tail
Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace standard XE with OaXE | CMLM · WMT14, WMT16, and WMT17 translation benchmarks | BLEU/SacreBLEU: 26.1 vs 18.3; 30.2 vs 22.0; 32.4 vs 27.6; 33.3 vs 28.6; 32.9 vs 23.6; 22.1 vs 13.5 | Helped |
| replace AXE with OaXE | CMLM · WMT14, WMT16, and WMT17 translation benchmarks | BLEU/SacreBLEU: 26.1 vs 23.5; 30.2 vs 27.9; 32.4 vs 30.8; 33.3 vs 31.5; 32.9 vs 30.9; 22.1 vs 19.8 | Helped |
| train with OaXE on raw rather than distilled data | CMLM · WMT14 En-De and De-En raw data | BLEU: 22.4 vs 10.6; 26.8 vs 15.1 | Helped |
| replace CMLM with OaXE-trained CMLM | CMLM · WAT17 Ja-En | BLEU: 20.9 -> 28.4 | Helped |
| add XE annealing initialization to OaXE (ablation) | CMLM + OaXE · WMT14 En-De and De-En validation | BLEU: 23.8; 28.9 | Helped |
| add XE pre-training initialization to OaXE (ablation) | CMLM + OaXE · WMT14 En-De and De-En validation | BLEU: 24.1; 29.4 | Helped |
| remove XE initialization (ablation) | CMLM + OaXE · WMT14 En-De validation | BLEU: 0 BLEU | Hurt |
| use early XE pre-training initialization (ablation) | CMLM + OaXE · WMT14 En-De validation | BLEU: 9.0 BLEU at 10K steps; 18.5 BLEU | Helped |
| truncate predictions below the confidence threshold (ablation) | CMLM + OaXE · WMT14 En-De validation | BLEU: +0.6 BLEU when margin increases from 0 to 0.15 | Mixed |
| use partially observed instead of unobserved inputs (ablation) | CMLM + OaXE · WMT14 En-De and De-En validation | BLEU: 24.1 vs 24.0; 29.4 vs 29.1 | No clear effect |
| predict only masked tokens rather than all tokens (ablation) | CMLM + OaXE · WMT14 En-De and De-En validation | BLEU: 24.0 vs 24.0; 29.1 vs 29.0 | No clear effect |
| add OaXE loss to CMLM (ablation) | CMLM · WMT14 En-De and De-En test, raw data | repeated token percentage: 22.80% -> 1.70%; 22.70% -> 1.39% | Helped |
Closest papers by meaning
- Aligned Cross Entropy for Non-Autoregressive Machine Translation (ICML 2020 · similarity 0.93)
- Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation (NeurIPS 2022 · similarity 0.88)
- AEQA-NAT : Adaptive End-to-end Quantization Alignment Training Framework for Non-autoregressive Machine Translation (ICML 2025 · similarity 0.86)
- On the Entropy Calibration of Language Models (NeurIPS 2025 · similarity 0.85)
- Confidence-Based Decoding is Provably Efficient for Diffusion Language Models (NeurIPS 2026 · similarity 0.84)
- Cut Your Losses in Large-Vocabulary Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.