Research map / Learning theory and algorithms / Kernels and regression theory
On the Downstream Performance of Compressed Word Embeddings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use eigenspace overlap score instead of Delta_max to predict downstream performance | compression quality measure · SQuAD (fastText) | abs Spearman correlation: 0.72 -> 0.91 | Helped |
| use eigenspace overlap score instead of PIP loss as selection criterion | compressed embedding selection · SQuAD (fastText) | selection error rate: 0.37 -> 0.11 | Helped |
| use eigenspace overlap score instead of Delta_max as selection criterion | compressed embedding selection · MNLI (BERT WordPiece) | selection error rate: 0.15 -> 0.10 | Helped |
| uniform quantization 32x instead of DCCL | GloVe compression · SQuAD | F1 below uncompressed: 0.47 below vs 0.43 below | No clear effect |
Closest papers by meaning
- Online Embedding Compression for Text Classification Using Low Rank Matrix Factorization (AAAI 2019 · similarity 0.91)
- NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models (ICLR 2025 · similarity 0.87)
- Compressing LLMs: The Truth is Rarely Pure and Never Simple (ICLR 2024 · similarity 0.87)
- LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression (NeurIPS 2026 · similarity 0.87)
- Accuracy is Not All You Need (NeurIPS 2024 · similarity 0.86)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.