Research map / Vision-language and multimodal / CLIP and zero-shot
Cross-Lingual Natural Language Generation via Pre-Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Xnlg pre-training instead of the XLM initialization alone | XLM-based model · Zero-shot Zh-Zh-QG | BLEU-4: 0.25 -> 16.37 | Helped |
| Use Xnlg rather than the strongest translation pipeline | Zero-shot Zh-Zh-QG · Zero-shot Zh-Zh-QG | BLEU-4: 9.95 -> 16.37 | Helped |
| Use Xnlg rather than the strongest translation pipeline | Zero-shot French abstractive summarization · Zero-shot Fr-AS | ROUGE-1: 38.48 -> 39.98 | Helped |
| Use Xnlg rather than the strongest translation pipeline | Zero-shot Chinese abstractive summarization · Zero-shot Zh-AS | ROUGE-1: 36.96 -> 41.66 | Helped |
| Add the DAE pre-training objective (ablation) | Xnlg · Zero-shot Zh-Zh-QG | BLEU-4: 0.38 -> 16.37 | Helped |
| Add the XAE pre-training objective (ablation) | Xnlg · Zero-shot Zh-Zh-QG | BLEU-4: 13.71 -> 16.37 | Helped |
| Fine-tune all parameters rather than encoder Transformer layers only (ablation) | Xnlg · Supervised En-En-QG | BLEU-4: 19.69 -> 19.99 | Helped |
| Fine-tune all parameters rather than encoder Transformer layers only (ablation) | Xnlg · Zero-shot Zh-Zh-QG | BLEU-4: 16.37 -> 6.82 | Hurt |
| Fine-tune encoder Transformer layers rather than decoder parameters only (ablation) | Xnlg · Zero-shot Zh-Zh-QG | BLEU-4: 0.21 -> 16.37 | Helped |
| Fine-tune encoder Transformer layers rather than decoder parameters only (ablation) | Xnlg · Supervised En-En-QG | BLEU-4: 16.37 -> 19.69 | Helped |
| Fine-tune only encoder parameters rather than decoder parameters only (ablation) | Xnlg · Zero-shot Zh-Zh-QG | BLEU-4: 0.21 -> 15.72 | Helped |
Closest papers by meaning
- Cross-lingual Language Model Pretraining (NeurIPS 2019 · similarity 0.90)
- Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework (ICLR 2020 · similarity 0.89)
- Cross-Lingual Pre-Training Based Transfer for Zero-Shot Neural Machine Translation (AAAI 2020 · similarity 0.88)
- Constrained Decoding for Cross-lingual Label Projection (ICLR 2024 · similarity 0.88)
- Enhancing Non-English Capabilities of English-Centric Large Language Models Through Deep Supervision Fine-Tuning (AAAI 2025 · similarity 0.87)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.