Research map / Large language models / AI and society
In Search of Coherence and Consensus: Measuring the Interpretability of Statistical Topics
Fred Morstatter, Huan Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Avg. Pairwise JCD as automated proxy for Model Precision | topic interpretability measurement · ERC abstracts corpus | Spearman rho: 1.000 | Improved |
| use TP-HHI as automated proxy for Topic Consensus | topic interpretability measurement · ERC abstracts corpus | Spearman rho: 0.885 | Improved |
| use TP-HHI as proxy for Topic Consensus | topic interpretability measurement · Yahoo News corpus | Spearman rho: 0.913 | Improved |
| use NPMI as proxy for Model Precision | topic interpretability measurement · ERC abstracts corpus | Spearman rho: -0.562 | Hurt |
| predict Topic Consensus instead of Model Precision Choose Two with linear regression on automated measures | topic interpretability prediction · ERC abstracts corpus | RMSE: 0.352 -> 0.131 | Improved |
Closest papers by meaning
- Is Automated Topic Model Evaluation Broken? The Incoherence of Coherence (NeurIPS 2021 · similarity 0.90)
- Diagnosing and Improving Topic Models by Analyzing Posterior Variability (AAAI 2018 · similarity 0.85)
- On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling (AAAI 2024 · similarity 0.84)
- Precision-Recall Balanced Topic Modelling (NeurIPS 2019 · similarity 0.84)
- BBScore: A Brownian Bridge Based Metric for Assessing Text Coherence (AAAI 2024 · similarity 0.83)
- Hierarchical Coherence Modeling for Document Quality Assessment (AAAI 2021 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
