Research map / Probabilistic and scientific modeling / Variational and generative sampling
Principled Selection of Hyperparameters in the Latent Dirichlet Allocation Model
Clint P. George, Hani Doss
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ST estimator I-hat_st (with extra stored averages) instead of I-tilde_st for posterior probability estimation (ablation) | Serial tempering on Wikipedia Cats corpus · Wikipedia Cats corpus, nu(||theta7-theta8||<=.07) | variability across independent runs: two independent I-hat estimates close; two I-tilde estimates not | Improved |
| Use serial tempering M-hat instead of M-tilde for marginal likelihood estimation (ablation) | Serial tempering · Synthetic and Wikipedia corpora | estimator variance | Improved |
| Use Serial tempering instead of VI-EM to estimate hyperparameter | LDA hyperparameter estimation · Synthetic corpora K=4,8; D=20,40,100 | accuracy of h estimate vs true argmax: VI-EM estimates far from true value and depend on starting values | Improved |
| Use Serial tempering instead of Gibbs-EM | LDA hyperparameter estimation · Synthetic corpora K=4,8; D=20,40,100 | accuracy of h estimate: Gibbs-EM points far from inside 95% confidence ellipse | Improved |
| Use Gibbs-EM instead of VI-EM | EM-based hyperparameter estimation · Synthetic corpora | accuracy of h estimate | Improved |
| Double Gibbs chain length each EM iteration (m_n = 2^(6+n)) (ablation) | Gibbs-EM · Synthetic corpora | accuracy of h estimate | No clear effect |
| Increase number of documents D from 20 to 100 | VI-EM and Gibbs-EM · Synthetic corpora | accuracy of h estimate | Improved |
| Use Gibbs-EM instead of VI-EM for speed | hyperparameter estimation · Synthetic corpora | computation time: time for Gibbs-EM about 55 times the time for VI-EM | Hurt |
| Use Serial tempering instead of Gibbs-EM for speed | hyperparameter estimation · Synthetic corpora | computation time: time for ST about seven times the time for Gibbs-EM | Hurt |
| Increase number of documents in synthetic corpus | Serial tempering argmax · Synthetic corpora | closeness of h-hat to h_true | Improved |
| Use Augmented Collapsed Gibbs Sampler vs Grouped Gibbs Sampler (ablation) | Serial tempering chain · Wikipedia Cats corpus | estimates | No clear effect |
| Use collapsed Gibbs sampler instead of grouped Gibbs sampler (ablation) | LDA posterior sampling · LDA mixing rate | mixing rate | Improved |
Closest papers by meaning
- A Statistical Approach for Optimal Topic Model Identification (JMLR 2022 · similarity 0.91)
- Online Bayesian Moment Matching for Topic Modeling with Unknown Number of Topics (NeurIPS 2016 · similarity 0.90)
- Pólya Urn Latent Dirichlet Allocation: A Doubly Sparse Massively Parallel Sampler (TPAMI 2018 · similarity 0.89)
- Dense Distributions from Sparse Samples: Improved Gibbs Sampling Parameter Estimators for LDA (JMLR 2017 · similarity 0.87)
- Discriminative Topic Modeling with Logistic LDA (NeurIPS 2019 · similarity 0.87)
- Large Language Bayes (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
