Research map / Probabilistic and scientific modeling / Molecules and proteins

On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling

AAAI 2024 · Probabilistic and scientific modeling › Molecules and proteins · approach: Probabilistic embeddings and relational inference · 9 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TraCo, a neural hierarchical topic model that uses sparse, balanced transport-plan dependencies (via Sinkhorn OT) and a context-aware disentangled decoder with contextual topical biases to improve affinity, rationality, and diversity of topic hierarchies.

Problem

Neural hierarchical topic models produce topic hierarchies with low affinity between parent and child topics, low rationality (child topics as general as parents), and low diversity among sibling topics.

Approach

Model parent-child topic dependencies as entropic optimal transport plans (solved by Sinkhorn) with marginal constraints so they are sparse and balanced, use them to regularize topic embeddings, and decode each level separately with a contextual topical bias built from neighboring levels' top words to separate granularity.

Key results

• On NeurIPS, TraCo reaches topic diversity 0.824 versus 0.632 for the runner-up HyperMiner, with coherence 0.438 vs 0.385 for the best baseline.
• On Wikitext-103 hierarchy quality, TraCo gets PCC 0.167 and SD 0.960, while removing TPD drops PCC to -0.286 and SD to 0.452.
• TraCo achieves PCC 0.077 and PCD 0.958 on NeurIPS versus 0.014 and 0.905 for the runner-up NGHTM, and outperforms baselines on downstream classification and clustering on NYT and 20NG.

Measured effects reported in the paper

Surprising findings

• Removing the context-aware disentangled decoder on Wikitext-103 raises PCC from 0.167 to 0.030 relative to... (not surprising); a notable one is that without TPD PCC turns negative on several datasets.

Limitations

• Relies on a fixed 3-level hierarchy with predefined topic counts per level (10, 50, 200).
• Several hyperparameters (tau, N_top, lambda_b, lambda_TPD, epsilon) are set manually without reported sensitivity analysis in the text.

Experimental setup

Datasets: NeurIPS ACL NYT Wikitext-103 20NG

Benchmarks: NeurIPS ACL NYT Wikitext-103 20NG

Metrics: topic coherence (C_V) topic diversity parent-child coherence (CLNPMI) parent-child diversity parent-non-child diversity sibling diversity classification accuracy F1 purity NMI

Baselines: nTSNTM HNTM NGHTM SawETM DCETM ProGBN HyperMiner

Builds on: HyperMiner ProGBN hLDA Sinkhorn algorithm

Closest papers by meaning

• TopicNet: Semantic Graph-Guided Topic Discovery (NeurIPS 2021 · similarity 0.88)
• Dirichlet belief networks for topic structure learning (NeurIPS 2018 · similarity 0.87)
• Knowledge-Aware Bayesian Deep Topic Model (NeurIPS 2022 · similarity 0.87)
• Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network (ICML 2021 · similarity 0.86)
• Topic Modeling Revisited: A Document Graph-based Neural Network Perspective (NeurIPS 2021 · similarity 0.86)
• FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model (NeurIPS 2024 · similarity 0.85)