Research map / Label-efficient and robust learning / Recognition and long-tail
Alternating Language Modeling for Cross-Lingual Pre-Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ALM code-switched pre-training to Transformer | Transformer · WMT14 En-De | BLEU: 28.40 -> 29.22 | Helped |
| add ALM code-switched pre-training to Transformer | Transformer · IWSLT14 De-En | BLEU: 34.49 -> 35.53 | Helped |
| ALM (code-switched) instead of XLM TLM pre-training | cross-lingual pre-training · WMT14 En-De | BLEU: 28.88 -> 29.22 | Helped |
| ALM instead of XLM pre-training | cross-lingual pre-training · IWSLT14 De-En | BLEU: 35.22 -> 35.53 | Helped |
| ALM instead of XLM pre-training | cross-lingual pre-training · XNLI TRANSLATE-TRAIN avg | accuracy: 76.7 -> 77.2 | Helped |
| ALM instead of XLM pre-training | cross-lingual pre-training · XNLI TRANSLATE-TEST avg | accuracy: 74.2 -> 74.6 | Helped |
| ALM instead of XLM pre-training | cross-lingual pre-training · XNLI CROSS-LINGUAL TEST avg | accuracy: 75.1 -> 75.6 | Helped |
| ALM pre-training with only 10% of parallel fine-tuning data | Transformer · IWSLT14 De-En low resource | BLEU: 18.78 -> 22.64 | Helped |
| ALM pre-training with full (100%) parallel data | Transformer · IWSLT14 De-En low resource | BLEU: 34.49 -> 35.53 | Helped |
Closest papers by meaning
- Cross-lingual Language Model Pretraining (NeurIPS 2019 · similarity 0.91)
- Cross-Lingual Natural Language Generation via Pre-Training (AAAI 2020 · similarity 0.88)
- Cross-model Control: Improving Multiple Large Language Models in One-time Training (NeurIPS 2024 · similarity 0.87)
- Enhancing Multilingual LLM Pretraining with Model-Based Data Selection (NeurIPS 2025 · similarity 0.86)
- In-Context Unlearning: Language Models as Few-Shot Unlearners (ICML 2024 · similarity 0.85)
- Constrained Decoding for Cross-lingual Label Projection (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.