Research map / Probabilistic and scientific modeling / Variational and generative sampling
Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| represent documents and topics as mixtures of word and topic embeddings and jointly minimize bidirectional conditional transport cost | WeTe · WS, RCV2, DP, 20NG(6) | km-Purity and km-NMI | Helped |
| fine-tune pretrained GloVe word embeddings (ablation) | WeTe · WS, RCV2, DP, 20NG(6) | km-Purity: 59.0$\pm$0.1 -> \textbf{60.8}$\pm$0.2; \underline{59.2}$\pm$0.2 -> \textbf{62.9}$\pm$0.5; \underline{75.8} $\pm$0.8 -> \ | Mixed |
| fine-tune pretrained GloVe word embeddings (ablation) | WeTe · WS, RCV2, DP, 20NG(6) | km-NMI: \underline{34.5}$\pm$0.1 -> \textbf{34.9}$\pm$0.4; 40.3$\pm$0.4 -> \textbf{42.8}$\pm$0.3; \underline{62.5}$\pm$0.8 -> \t | Mixed |
| learn word embeddings from a Gaussian initialization instead of loading pretrained GloVe (ablation) | WeTe · WS, RCV2, DP, 20NG(6) | km-Purity: 59.0$\pm$0.1 -> \underline{59.7}$\pm$0.1; \underline{59.2}$\pm$0.2 -> 58.5$\pm$0.3; \underline{75.8} $\pm$0.8 -> 74.1 $\ | Mixed |
| learn word embeddings from a Gaussian initialization instead of loading pretrained GloVe (ablation) | WeTe · WS, RCV2, DP, 20NG(6) | km-NMI: 34.5 -> 34.1; 40.3 -> 41.2; 62.5 -> 60.1; 35.0 -> 34.3 | Mixed |
| use only the conditional transport cost, removing the Poisson likelihood objective (ablation) | WeTe · 20NG | topic quality | Helped |
| use only the Poisson likelihood, removing conditional transport cost (ablation) | WeTe · 20NG | document representation | Helped |
| combine conditional transport cost with Poisson likelihood rather than using either alone (ablation) | WeTe · 20NG | topic quality and document representation | Helped |
Closest papers by meaning
- Knowledge-Aware Bayesian Deep Topic Model (NeurIPS 2022 · similarity 0.89)
- Context-guided Embedding Adaptation for Effective Topic Modeling in Low-Resource Regimes (NeurIPS 2023 · similarity 0.89)
- Bayesian Deep Embedding Topic Meta-Learner (ICML 2022 · similarity 0.88)
- textTOvec: DEEP CONTEXTUALIZED NEURAL AUTOREGRESSIVE TOPIC MODELS OF LANGUAGE WITH DISTRIBUTED COMPOSITIONAL PRIOR (ICLR 2019 · similarity 0.88)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.87)
- Contextual Document Embeddings (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.