Research map / Probabilistic and scientific modeling / Variational and generative sampling
A Statistical Approach for Optimal Topic Model Identification
Craig M. Lewis, Francesco Grossetti
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OpTop chi-square test instead of perplexity for choosing topic count | LDA topic count selection · Simulated corpora (Ksim=10) | proportion closer to true K: 0.84 | Improved |
| use OpTop instead of perplexity for choosing topic count | LDA topic count selection · Simulated corpora (Ksim=15) | proportion closer to true K: 0.90 | Improved |
| use OpTop instead of perplexity for choosing topic count | LDA topic count selection · Simulated corpora (Ksim=20 to 35) | proportion closer to true K: 1.00 | Improved |
| use OpTop instead of perplexity on the inaugural corpus | LDA topic count selection · US Presidential Inaugural Addresses | selected number of topics: 86 -> 92 | Mixed |
| add more topics beyond the optimal 92 (93 to 200) (ablation) | OpTop-selected 92-topic LDA · US Presidential Inaugural Addresses | topic stability (scaled TWW chi-square): scaled means approach 1.0 versus cutoff 1.414 | No clear effect |
| add uninformative least-similar topics beyond K=92 (ablation) | OpTop-selected 92-topic LDA · US Presidential Inaugural Addresses | incremental explanatory power | No clear effect |
Closest papers by meaning
- Principled Selection of Hyperparameters in the Latent Dirichlet Allocation Model (JMLR 2018 · similarity 0.91)
- Online Bayesian Moment Matching for Topic Modeling with Unknown Number of Topics (NeurIPS 2016 · similarity 0.90)
- Pólya Urn Latent Dirichlet Allocation: A Doubly Sparse Massively Parallel Sampler (TPAMI 2018 · similarity 0.89)
- A Reduction for Efficient LDA Topic Reconstruction (NeurIPS 2018 · similarity 0.88)
- Choosing the Number of Topics in LDA Models – A Monte Carlo Comparison of Selection Criteria (JMLR 2024 · similarity 0.88)
- Discriminative Topic Modeling with Logistic LDA (NeurIPS 2019 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
