Research map / Architectures and efficiency / RNNs and normalization
ML-Embed: Inclusive and Efficient Embeddings for a Multilingual World
Ziyin Zhang, Zihan Liao, Hang Yu, Peng Di, Rui Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 3D-ML training before pruning to 120M (ablation) | EuroBERT-210M · MTEB average (17 benchmarks) | mean score: 44.10 -> 56.77 | Improved |
| add 3D-ML training before pruning to 120M (ablation) | EuroBERT-210M · MTEB Multilingual | mean score: 46.08 -> 56.94 | Improved |
| add MEL (nested-rank training) over baseline when compressing embedding at inference (ablation) | Qwen3-0.6B embedding model · MTEB English | mean score: 53.25 -> 64.30 | Improved |
| train with factorized embedding (decomposition-only, rank 512) versus original embedding (ablation) | Qwen3-0.6B embedding model · MTEB English | mean score: 69.68 -> 69.60 | No clear effect |
| add MEL on top of MLL (4 layers vs 1-layer baseline at same ~170M parameters) (ablation) | Qwen3-0.6B embedding model · MTEB English | mean score: +15 points | Improved |
| add MLL (single multi-exit model) versus individually trained shallow models (ablation) | Qwen3-0.6B embedding model · MTEB English | mean score | Hurt |
| use ML-Embed curated data instead of KaLM-Embedding data (ablation) | ML-Embed-0.6B · 17 MTEB benchmarks | benchmarks won: 9 out of 17 benchmarks | Improved |
| use ML-Embed curated data instead of KaLM-Embedding data (ablation) | ML-Embed-0.6B · MTEB Chinese | mean score | Hurt |
| scale model from 140M to 8B | ML-Embed · MTEB average | mean score: 50.76 -> 70.24 | Improved |
| compare ML-Embed-8B with previous top-1 on Polish | ML-Embed-8B · MTEB Polish | mean score: 50.95 -> 73.84 | Improved |
| compare ML-Embed-8B with previous top-1 on Multilingual/English | ML-Embed-8B · MTEB Multilingual | mean score: 72.32 -> 66.79 | Hurt |
| compare ML-Embed-0.6B with Qwen3-Embedding-0.6B | ML-Embed-0.6B · MTEB Multilingual | mean score: 64.34 -> 61.30 | Hurt |
Closest papers by meaning
- MMTEB: Massive Multilingual Text Embedding Benchmark (ICLR 2025 · similarity 0.92)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.91)
- The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embedding (NeurIPS 2024 · similarity 0.90)
- EmbedLLM: Learning Compact Representations of Large Language Models (ICLR 2025 · similarity 0.89)
- Repetition Improves Language Model Embeddings (ICLR 2025 · similarity 0.89)
- Scaling Embedding Layers in Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
