Research map / Learning theory and algorithms / Optimal transport
Distilled Wasserstein Learning for Word Embedding and Topic Modeling
Hongteng Xu, Wenlin Wang, Wei Liu, Lawrence Carin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use distilled learning with the smoothed underlying distance rather than no distillation (ablation) | DWL topic-model updates · Admission-type prediction training | training accuracy convergence | Improved |
| Use very heavy distillation (τ=0.25) instead of the selected τ=0.5 (ablation) | DWL topic-model updates · Admission-type prediction training | converged training accuracy | Hurt |
| Train DWL from scratch instead of fine-tuning Word2Vec embeddings (ablation) | DWL · MIMIC-III mortality and admission-type classification | accuracy: 71.01$\pm$0.12 vs 71.52$\pm$0.07; 74.74$\pm$0.11 vs 75.44$\pm$0.07; 84.54$\pm$0.13 vs 85.54$\pm$0.09; 89.49$\pm$0.12 vs | Mixed |
| Train DWL from scratch instead of fine-tuning Word2Vec embeddings (ablation) | DWL · MIMIC-III procedure recommendation | Top-5 F1: 30.81 vs 34.28 | Hurt |
| Fine-tune Word2Vec embeddings with DWL rather than use Word2Vec alone | MIMIC-III procedure recommendation · Top-5 recommendation | F1: 32.59 -> 34.28 | Improved |
| Represent documents with average-pooled embeddings and Euclidean distance rather than Wasserstein distance between word distributions | DWL · MIMIC-III mortality and admission-type classification | accuracy: 71.52$\pm$0.07 vs 70.65$\pm$0.07; 75.44$\pm$0.07 vs 76.00$\pm$0.06; 85.54$\pm$0.09 vs 84.35$\pm$0.08; 89.28$\pm$0.09 vs | No clear effect |
Closest papers by meaning
- Learning Embeddings into Entropic Wasserstein Spaces (ICLR 2019 · similarity 0.87)
- WASD: Wasserstein-based Knowledge Distillation for Large Language Models (NeurIPS 2026 · similarity 0.84)
- Tree-Wasserstein Distance for High Dimensional Data with a Latent Feature Hierarchy (ICLR 2025 · similarity 0.84)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.83)
- Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models (NeurIPS 2025 · similarity 0.83)
- FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
