Research map / Large language models / Language, safety and interpretability
Document Informed Neural Autoregressive Topic Models with Distributional Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add full left-and-right context to DocNADE | DocNADE on 15 datasets · 15 datasets, Avg (all) | PPL: 426 -> 406 | Helped |
| Add an embedding prior to the context-informed model (ablation) | iDocNADE on 15 datasets · 15 datasets, Avg (all) | PPL: 406 -> 404 | Helped |
| Add full context to DocNADE | DocNADE on 15 datasets · 15 datasets, Avg (all) | IR precision at retrieval fraction 0.02: .54 -> .58 | Helped |
| Add an embedding prior to iDocNADE (ablation) | iDocNADE on 15 datasets · 15 datasets, Avg (all) | IR precision at retrieval fraction 0.02: .58 -> .60 | Helped |
| Add an embedding prior to DocNADE (ablation) | DocNADE on 15 datasets · 15 datasets, Avg (all) | F1: .631 -> .661 | Helped |
| Add full context to DocNADE | DocNADE on 15 datasets · 15 datasets, Avg (all) | F1: .631 -> .645 | Helped |
| Add an embedding prior to iDocNADE (ablation) | iDocNADE on 15 datasets · 15 datasets, Avg (all) | F1: .645 -> .664 | Helped |
| Add full context to DocNADE | DocNADE on long-text datasets · Long-text datasets, Avg | Topic coherence W20: .602 -> .636 | Helped |
| Add an embedding prior to iDocNADE (ablation) | iDocNADE on short-text datasets · Short-text datasets, Avg | Topic coherence W20: .845 -> .847 | Helped |
| Add full context and embedding prior to DeepDNE | DeepDNE on 11 datasets · 11 datasets, Avg (all), deep models | PPL: 450 -> 402 | Helped |
| Add full context and embedding prior to DeepDNE | DeepDNE on 11 datasets · 11 datasets, Avg (all), deep models | IR precision at retrieval fraction 0.02: .59 -> .64 | Helped |
| Add an embedding prior to iDeepDNE (ablation) | iDeepDNE on 11 datasets · 11 datasets, Avg (all), deep models | PPL: 404 -> 402 | Helped |
Closest papers by meaning
- textTOvec: DEEP CONTEXTUALIZED NEURAL AUTOREGRESSIVE TOPIC MODELS OF LANGUAGE WITH DISTRIBUTED COMPOSITIONAL PRIOR (ICLR 2019 · similarity 0.93)
- Neural Dynamic Focused Topic Model (AAAI 2023 · similarity 0.88)
- Discovering Discrete Latent Topics with Neural Variational Inference (ICML 2017 · similarity 0.88)
- A Discrete Variational Recurrent Topic Model without the Reparametrization Trick (NeurIPS 2020 · similarity 0.88)
- Knowledge-Aware Bayesian Deep Topic Model (NeurIPS 2022 · similarity 0.88)
- Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided Document Generation (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.