Research map / Label-efficient and robust learning / Self-supervised and contrastive
Self-supervised and Supervised Joint Training for Resource-rich Machine Translation
Yong Cheng, Wei Wang, Lu Jiang, Wolfgang Macherey
This paper tried a common change: Add dropout
apply attention dropout and use model predictions in crossover-label computation · WMT'14 En→De · BLEU: 29.87 -> 30.46 Improved
What 795 accepted papers report for this change
- 549 improved
- 87 no clear effect
- 12 mixed
- 147 hurt
- Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference ICML 2026 · apply layer dropout to whole transformer blocks instead of attention and FFN sub-blocks separately · 20 TPP val loss, 271M · val loss: 2.377 -> 2.367Improved
- Quantifying the Uncertainty of Foundation Models with Singular Value Ensembles ICML 2026 · use MC Dropout · vision and SST-2 · accuracy: 86.3 -> 76.6 on Flowers102Hurt
- Motion Planning in Compressed Representation Spaces ICML 2026 · add nested dropout during training · WOMD guided generation · goal FDE: 4.145 -> 1.104Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| jointly train with F2-XEnDec rather than the reproduced Transformer | Transformer-big · WMT'14 En→De | BLEU: 30.88 -> 31.60 | Improved |
| jointly train with F2-XEnDec rather than the reproduced Transformer | Transformer-big · WMT'14 De→En | BLEU: 33.12 -> 34.94 | Improved |
| jointly train with F2-XEnDec rather than Joint Training with BART | Transformer-big · WMT'14 En→Fr | BLEU: 44.18 -> 45.15 | Improved |
| jointly train with F2-XEnDec rather than the reproduced Transformer | Transformer-big · WMT'14 Fr→En | BLEU: 39.82 -> 41.60 | Improved |
| add the instance-level fusion loss L_F2 (ablation) | F2-XEnDec · WMT'14 En→De | BLEU: 29.21 -> 30.46; -1.25 BLEU points | Improved |
| add the monolingual self-supervised loss L_F1 (ablation) | F2-XEnDec · WMT'14 En→De | BLEU: 29.55 -> 30.46 | Improved |
| use XEnDec with both parallel and monolingual inputs instead of parallel-only XEnDec (ablation) | F2-XEnDec · WMT'14 En→De | BLEU: 29.23 -> 30.46 | Improved |
| use crossover mixing rather than Mixup for the second XEnDec (ablation) | F2-XEnDec · WMT'14 En→De | BLEU: 29.67 -> 30.46 | Improved |
| apply attention dropout and use model predictions in crossover-label computation (ablation) | F2-XEnDec · WMT'14 En→De | BLEU: 29.87 -> 30.46 | Improved |
| use model predictions when computing crossover labels (ablation) | F2-XEnDec · WMT'14 En→De | BLEU: 30.24 -> 30.46 | Improved |
| increase monolingual data from size ×5 to ×10 (ablation) | F2-XEnDec · WMT'14 En→De | BLEU: 30.46 -> 30.22 | Hurt |
| increase monolingual data from size ×1 to ×5 (ablation) | F2-XEnDec · WMT'14 En→De | BLEU: 29.84 -> 30.46 | Improved |
Closest papers by meaning
- Unsupervised Neural Machine Translation (ICLR 2018 · similarity 0.88)
- Joint Training for Neural Machine Translation Models with Monolingual Data (AAAI 2018 · similarity 0.88)
- Deep Fusing Pre-trained Models into Neural Machine Translation (AAAI 2022 · similarity 0.87)
- Dual Learning for Machine Translation (NeurIPS 2016 · similarity 0.87)
- MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods (ICLR 2024 · similarity 0.86)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
