Research map / Architectures and efficiency / Attention and transformers
Neural Machine Translation with Joint Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Transformer with Reformer-base | NMT translation · IWSLT14 De-En test | BLEU: +0.8 BLEU point | Helped |
| replace Transformer with Reformer-fast | NMT translation · IWSLT15 Vi-En test | BLEU: +1.3 BLEU point | Helped |
| replace Transformer with Reformer-base | NMT translation · IWSLT15 Vi-En | BLEU | Mixed |
| replace Transformer with Reformer variants | NMT translation · NIST12 Zh-En | BLEU: +1.9 BLEU point | Helped |
| add Dropout 1/2d (ablation) | Reformer · IWSLT14 De-En valid | BLEU: 35.16 -> 35.87; 0.7 BLEU point improvement | Helped |
| add Dropout 1/2d (ablation) | Reformer · IWSLT14 De-En valid | PPL: 5.00 -> 4.82 | Helped |
| add PreNet (ablation) | Reformer-base · IWSLT14 De-En valid | BLEU: 35.16 -> 35.87 | No clear effect |
| add PreNet (ablation) | Reformer-base · IWSLT14 De-En decoding | speed: 0.47× -> 0.73× | Helped |
| apply gradient-based scaling | Reformer-fast · IWSLT14 De-En test | BLEU: +0.7 BLEU point | Helped |
| apply gradient-based scaling | Reformer-fast · NIST12 Zh-En MT08 | BLEU: +2 BLEU points | Helped |
Closest papers by meaning
- Joint Training for Neural Machine Translation Models with Monolingual Data (AAAI 2018 · similarity 0.90)
- From Fully Trained to Fully Random Embeddings: Improving Neural Machine Translation with Compact Word Embedding Tables (AAAI 2022 · similarity 0.90)
- Deep Fusing Pre-trained Models into Neural Machine Translation (AAAI 2022 · similarity 0.90)
- Neural Machine Translation Advised by Statistical Machine Translation (AAAI 2017 · similarity 0.89)
- Improved Neural Machine Translation with SMT Features (AAAI 2016 · similarity 0.89)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.