Research map / Large language models / Language, safety and interpretability
textTOvec: DEEP CONTEXTUALIZED NEURAL AUTOREGRESSIVE TOPIC MODELS OF LANGUAGE WITH DISTRIBUTED COMPOSITIONAL PRIOR
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add an LSTM language-model context to DocNADE | DocNADE · All datasets (Avg) | Topic coherence W20: .755 -> .772 | Helped |
| Add pretrained word-embedding priors to ctx-DocNADE | ctx-DocNADE · All datasets (Avg) | Topic coherence W20: .755 -> .790 | Helped |
| Add LSTM context and embedding priors | DocNADE(RV) · All datasets | IR precision at 0.02: .577 -> .615 | Helped |
| Add LSTM context and embedding priors | DocNADE(RV) · All datasets | F1: .634 -> .662 | Helped |
| Add LSTM context to DocNADE | DocNADE · AGnewstitle | Perplexity: 846 -> 822 | Helped |
| Add embedding priors to ctx-DocNADE | ctx-DocNADE · BNC | Topic coherence NPMI | No clear effect |
| Use the deep model variant with embeddings | ctx-DocNADEe · Short-text datasets | IR precision at 0.02: .606 (TREC6); .878 (Subjectivity) | Mixed |
| Set the LM mixture weight λ to 0.1 rather than 1.0 (ablation) | ctx-DocNADE · 20NSshort | Perplexity: 899.04 -> 829.5 | Helped |
| Set the LM mixture weight λ to 0.1 rather than 1.0 (ablation) | ctx-DocNADEe · 20NSshort | Perplexity: 890.3 -> 828.8 | Helped |
| Set the LM mixture weight λ to 0.1 rather than 1.0 (ablation) | ctx-DocNADE · TMNtitle | Perplexity: 1898.1 -> 1482.7 | Helped |
| Set the LM mixture weight λ to 0.5 rather than 1.0 (ablation) | ctx-DocNADEe · 20NSshort | IR precision at 0.02: 0.277 -> 0.278 | Helped |
| Vary the LM mixture weight λ across 1.0, 0.8, 0.5, and 0.3 (ablation) | ctx-DocNADE · R21578 | IR precision at 0.02: 0.714, 0.714, 0.714, 0.714 | No clear effect |
Closest papers by meaning
- Document Informed Neural Autoregressive Topic Models with Distributional Prior (AAAI 2019 · similarity 0.93)
- TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency (ICLR 2017 · similarity 0.89)
- Recurrent Hierarchical Topic-Guided RNN for Language Generation (ICML 2020 · similarity 0.89)
- Context-guided Embedding Adaptation for Effective Topic Modeling in Low-Resource Regimes (NeurIPS 2023 · similarity 0.88)
- Improving Topic Modeling by Distilling Soft Labels from Language Models (ICML 2026 · similarity 0.88)
- Knowledge-Aware Bayesian Deep Topic Model (NeurIPS 2022 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.