Research map / Probabilistic and scientific modeling / Variational and generative sampling
Deep Autoencoding Topic Model With Scalable Hybrid Bayesian Inference
Hao Zhang, Bo Chen, Yulai Cong, Dandan Guo, Hongwei Liu, Mingyuan Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Weibull reparameterization instead of gamma approximation (RSVI) (ablation) | DATM · 20News perplexity | perplexity: 604 -> 581 | Improved |
| use stochastic-gradient MCMC (TLASGR) for global parameters instead of plain SGD (ablation) | DATM-WAI · 20News perplexity | perplexity: 581 -> 581 (20News); 984 -> 980 (Wiki) | Improved |
| add stochastic downward path to inference network (ablation) | DATM-IWHAI · 20News perplexity | perplexity: 588 -> 581 | Improved |
| add depth: 128 to 128-64 to 128-64-32 layers (ablation) | DATM-WHAI · 20News perplexity | perplexity: 591 -> 582 -> 581 | Improved |
| add depth to DATM-IWHAI (no downward path) (ablation) | DATM-IWHAI · 20News perplexity | perplexity: 592 -> 589 -> 588 | No clear effect |
| replace logistic-normal AVITM latent with Weibull in single-layer model | AVITM · 20News perplexity | perplexity: 654 -> 591 | Improved |
| learn layer widths via layer-wise training with K1max=128 | DATM-WHAI · 20News perplexity | perplexity: 581 -> 578 | Improved |
| increase K1max from 64 to 1024 (ablation) | DATM-WHAI · 20News perplexity | perplexity: 584 -> 573 | Improved |
| use hybrid inference with TLASGR instead of Gibbs sampling at test time | DLDA-Gibbs · 20News test time | seconds per document: 10.46 -> 0.63 | Improved |
| add nonlinearity from latent features to labels (sDATM-N vs sDATM-L) (ablation) | sDATM · 20News classification | error rate (%): 18.63 -> 15.81 | Improved |
| add nonlinearity (sDATM-N vs sDATM-L) (ablation) | sDATM · MNIST | error rate (%): 1.03 -> 0.97 | Improved |
| add label supervision to DATM | DATM · IMDB classification | error rate (%): 20.42 -> 10.92 | Improved |
Closest papers by meaning
- WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling (ICLR 2018 · similarity 0.92)
- Generative Text Convolutional Neural Network for Hierarchical Document Representation Learning (TPAMI 2022 · similarity 0.88)
- Autoencoding Variational Inference For Topic Models (ICLR 2017 · similarity 0.88)
- Bayesian Progressive Deep Topic Model with Knowledge Informed Textual Data Coarsening Process (ICML 2023 · similarity 0.88)
- Alleviating "Posterior Collapse'' in Deep Topic Models via Policy Gradient (NeurIPS 2022 · similarity 0.88)
- Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided Document Generation (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
