Research map / Label-efficient and robust learning / Recognition and long-tail
Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the massively multilingual NMT encoder instead of mBERT | English-trained XNLI classifier · XNLI zero-shot, average across 15 languages | accuracy: 66.3 -> 67.5 | Helped |
| Use the massively multilingual NMT encoder instead of mBERT | MLDoc classifier · MLDoc zero-shot, average across 8 languages | accuracy: 74.5 -> 74.7 | Helped |
| Use the massively multilingual NMT encoder instead of mBERT | Intent classifier · Intent classification zero-shot, average across 3 languages | accuracy: 70.6 -> 94.2 | Helped |
| Use the massively multilingual NMT encoder instead of mBERT | POS tagger · POS zero-shot, 48 languages | F1: 74.87 -> 75.43 | Helped |
| Use the massively multilingual NMT encoder instead of mBERT | NER tagger · NER zero-shot, average across 4 languages | span-based F1: 78.5 -> 60.6 | Hurt |
| Freeze the encoder and train only the task-specific layer | MMTE POS tagger · POS zero-shot, 48 languages, feature-based | F1: 74.55 -> 78.52 | Helped |
| Add 10 labeled examples from each target language alongside English data | MMTE POS tagger · POS few-shot, 48 languages | F1: 83.97 -> 84.58 | Helped |
| Use one jointly trained model for all languages rather than a separate model per language | MMTE POS tagger · POS in-language, 48 languages | F1: 95.49 -> 95.69 | Helped |
| Increase pre-training coverage from 4 to 102 languages (ablation) | Massively multilingual NMT encoder · POS zero-shot, English | F1: 96.30 -> 96.28 | Hurt |
| Increase pre-training coverage from 4 to 102 languages (ablation) | Massively multilingual NMT encoder · XNLI zero-shot, English | accuracy: 80.1 -> 79.6 | Hurt |
| Remove the target-language token from the source input and encode it separately (ablation) | 4-language MMTE · POS zero-shot, average | F1: 89.6 -> 90.5 | Helped |
| Remove the target-language token from the source input and encode it separately (ablation) | 4-language MMTE · XNLI zero-shot, average | accuracy: 74.6 -> 75.4 | Helped |
Closest papers by meaning
- Cross-Lingual Pre-Training Based Transfer for Zero-Shot Neural Machine Translation (AAAI 2020 · similarity 0.91)
- XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalisation (ICML 2020 · similarity 0.90)
- Unsupervised Neural Machine Translation (ICLR 2018 · similarity 0.89)
- Multilingual Neural Machine Translation With Soft Decoupled Encoding (ICLR 2019 · similarity 0.89)
- Scaling Laws for Multilingual Neural Machine Translation (ICML 2023 · similarity 0.89)
- Constrained Decoding for Cross-lingual Label Projection (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.