Research map / Probabilistic and scientific modeling / Variational and generative sampling
Discovering Discrete Latent Topics with Neural Variational Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the explicit topic-mixture decoder with an implicit topic distribution (ablation) | GSM, GSB, and RSB neural models · MXM, 20NewsGroups, and RCV1 test sets | perplexity: 306 -> 270 (GSM, MXM, 50); 822 -> 787 (GSM, 20NewsGroups, 50); 717 -> 653 (GSM, RCV1, 50) | Helped |
| Add topic diversity regularisation (ablation) | neural topic models · Perplexity evaluation across datasets | perplexity: 2∼5 | No clear effect |
| Add topic diversity regularisation (ablation) | neural topic models · Topic redundancy | topic redundancy | Helped |
| Use recurrent stick-breaking topic proportions instead of GSM or GSB | finite neural topic models · 20NewsGroups, 50 and 200 topics | perplexity: 822 -> 835 (50 topics, GSM to RSB); 830 -> 822 (200 topics, GSM to RSB) | Mixed |
| Increase the maximum topic count beyond 400 | GSM · 20NewsGroups test set | perplexity | Hurt |
| Use stick breaking rather than GSM when the maximum topic count greatly exceeds need | GSB and RSB · 20NewsGroups test set | perplexity | Helped |
| Allow RSB-TF to dynamically add active topics | RSB-TF · 20NewsGroups | training perplexity | Helped |
| Use an implicit rather than explicit topic distribution (ablation) | neural document models · 20NewsGroups, 50 dimensions | topic coherence (NPMI): 0.121 -> 0.223 (GSM) | Helped |
Closest papers by meaning
- A Discrete Variational Recurrent Topic Model without the Reparametrization Trick (NeurIPS 2020 · similarity 0.94)
- Neural Dynamic Focused Topic Model (AAAI 2023 · similarity 0.90)
- Document Informed Neural Autoregressive Topic Models with Distributional Prior (AAAI 2019 · similarity 0.88)
- TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency (ICLR 2017 · similarity 0.88)
- textTOvec: DEEP CONTEXTUALIZED NEURAL AUTOREGRESSIVE TOPIC MODELS OF LANGUAGE WITH DISTRIBUTED COMPOSITIONAL PRIOR (ICLR 2019 · similarity 0.88)
- Topic Modeling Revisited: A Document Graph-based Neural Network Perspective (NeurIPS 2021 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.