Research map / Label-efficient and robust learning / Recognition and long-tail
Empirical Regularization for Synthetic Sentence Pairs in Unsupervised Neural Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add GAN regularization to synthetic sentence pairs | MASS, XLM, and mBART · De↔En, Fr↔En, Ro↔En newstest; individual directions shown in Table 2 | BLEU: 34.91 -> 36.04; 28.03 -> 28.98; 34.42 -> 35.61; 37.02 -> 38.29; 32.75 -> 34.01; 34.82 -> 35.91 (MASS) | Helped |
| Add GAN regularization to synthetic sentence pairs | XLM and mBART · De↔En, Fr↔En, Ro↔En newstest; individual directions shown in Table 2 | BLEU: 33.81 -> 34.94; 26.32 -> 27.12; 32.87 -> 33.92; 32.94 -> 34.24; 31.12 -> 32.53; 32.81 -> 34.01 (XLM); 33.65 -> 34.94; 29 | Helped |
| Add GAN regularization | 4-layer Transformer UNMT with UBWE pre-training · De↔En and Fr↔En test sets | BLEU: 21.34 -> 24.12; 17.89 -> 19.87; 24.20 -> 27.24; 25.83 -> 30.53 | Helped |
| Add BOW rather than GAN regularization | 4-layer Transformer UNMT with UBWE pre-training · Ru↔En newstest2016 | BLEU: 9.19 -> 10.42; 8.08 -> 9.35 | Helped |
| Use word-wise regularization instead of sentence-wise regularization | UBWE-pretrained UNMT · Similar pairs Fr↔En and De↔En | BLEU | Mixed |
| Anneal the regularization weight from 0 to 1 over the first 200k iterations (ablation) | Randomly initialized UNMT · De↔En | BLEU: 21.61 -> 22.81; 17.96 -> 18.62 | Helped |
| Use regularization with random initialization rather than no regularization (ablation) | Randomly initialized UNMT · De↔En | BLEU: 20.99 -> 21.61; 17.01 -> 17.96 | Helped |
| Vary λ for GAN regularization; λ = 1 versus other tested weights (ablation) | UBWE-pretrained 4-layer Transformer UNMT · De↔En | BLEU: 21.34; 23.89; 24.12; 22.12; 22.95; 23.87; 23.51; 22.30 (De→En); 17.89; 19.71; 19.87; 18.05; 18.31; 19.66; 19.85; 18.87 ( | Mixed |
| Use BPE rather than word tokenization (ablation) | GAN- and BOW-regularized UNMT · De↔En | BLEU: 33.04 -> 34.21 (Word-ENDE GAN); 33.81 -> 34.94 (BPE-ENDE GAN); 21.12 -> 23.94 (Word-UBWE GAN); 21.34 -> 24.12 (BPE-UBWE | Mixed |
| Add auto-encoder regularization | UBWE-pretrained UNMT · De↔En, Fr↔En, Ru↔En test sets | BLEU: 21.34 -> 23.46; 17.89 -> 18.91; 24.20 -> 26.57; 25.83 -> 29.55; 9.19 -> 10.11; 8.08 -> 9.04 | Helped |
| Add naive regularization | UBWE-pretrained UNMT · De↔En, Fr↔En, Ru↔En test sets | BLEU: 21.34 -> 23.01; 17.89 -> 18.57; 24.20 -> 26.01; 25.83 -> 28.51; 9.19 -> 9.42; 8.08 -> 8.31 | Helped |
Closest papers by meaning
- Unsupervised Neural Machine Translation (ICLR 2018 · similarity 0.90)
- Unsupervised Machine Translation Using Monolingual Corpora Only (ICLR 2018 · similarity 0.89)
- Controlling Neural Machine Translation Formality with Synthetic Supervision (AAAI 2020 · similarity 0.89)
- Regularizing Neural Machine Translation by Target-Bidirectional Agreement (AAAI 2019 · similarity 0.88)
- On Learning Language-Invariant Representations for Universal Machine Translation (ICML 2020 · similarity 0.87)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.