Research map / Label-efficient and robust learning / Recognition and long-tail
On Learning Universal Representations Across Languages
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply hierarchical sentence- and word-level contrastive learning to XLM-R | XLM-R · XTREME zero-shot transfer | Avg.: 68.2 -> 69.6 | Helped |
| Apply hierarchical sentence- and word-level contrastive learning to XLM-R | XLM-R · WMT'14 En→De | BLEU: 30.91 -> 31.74 | Helped |
| Apply hierarchical sentence- and word-level contrastive learning to XLM-R | XLM-R · WMT'14 En→Fr | BLEU: 43.27 -> 43.95 | Mixed |
| Add hard negative samples through smoothed linear interpolation | HiCTL · XTREME translate-train-all | Avg.: 73.2 -> 74.8 | Helped |
| Add sentence-level contrastive learning (ablation) | HiCTL · XTREME translate-train-all | Avg.: 72.4 -> 74.8 | Helped |
| Add word-level contrastive learning (ablation) | HiCTL · XTREME translate-train-all | Avg.: 74.2 -> 74.8 | Helped |
| Use parallel machine-translation data during pretraining (ablation) | HiCTL · XTREME translate-train-all | Tatoeba accuracy: 66.8 -> 69.1 | Helped |
| Use parallel machine-translation data during pretraining (ablation) | HiCTL · XTREME translate-train-all | Avg.: 73.8 -> 74.8 | Helped |
| Add hard negative samples | HiCTL · Tatoeba | Accuracy: 74.7 -> 81.3 | Helped |
| Use only 0.3GB of Gujarati monolingual pretraining data | HiCTL · Gujarati→English transfer after Hindi→English fine-tuning | BLEU: 0.0 -> 0.1 | Hurt |
Closest papers by meaning
- EMMA-X: An EM-like Multilingual Pre-training Algorithm for Cross-lingual Representation Learning (NeurIPS 2023 · similarity 0.91)
- Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework (ICLR 2020 · similarity 0.89)
- XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalisation (ICML 2020 · similarity 0.89)
- Soft Token Alignment for Cross-Lingual Reasoning (NeurIPS 2026 · similarity 0.86)
- Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model (AAAI 2024 · similarity 0.86)
- MCL-NER: Cross-Lingual Named Entity Recognition via Multi-View Contrastive Learning (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.