Research map / Architectures and efficiency / RNNs and normalization
TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LDA features to the RNN | RNN · Penn Treebank, 10 neurons | test perplexity: 225.0 -> 187.4 | Helped |
| Add TopicRNN's learned topic context | RNN · Penn Treebank, 10 neurons | test perplexity: 225.0 -> 172.2 | Helped |
| Use a GRU cell instead of a standard RNN cell | TopicRNN · Penn Treebank, 10 neurons | test perplexity: 172.2 -> 166.7 | Helped |
| Add LDA features to the RNN | RNN · Penn Treebank, 100 neurons | test perplexity: 142.1 -> 126.4 | Helped |
| Add TopicRNN's learned topic context | RNN · Penn Treebank, 100 neurons | test perplexity: 142.1 -> 122.3 | Helped |
| Use a GRU cell instead of a standard RNN cell | TopicRNN · Penn Treebank, 100 neurons | test perplexity: 122.3 -> 112.4 | Helped |
| Add TopicRNN's learned topic context | RNN · Penn Treebank, 300 neurons | test perplexity: 124.7 -> 112.2 | Helped |
| Use a GRU cell instead of a standard RNN cell | TopicRNN · Penn Treebank, 300 neurons | test perplexity: 112.2 -> 97.3 | Helped |
| Use an LSTM cell instead of a standard RNN cell | TopicRNN · Penn Treebank, 300 neurons | test perplexity: 112.2 -> 99.5 | Helped |
| Use unsupervised TopicRNN features for sentiment classification | IMDB sentiment classifier · IMDB 100k | error rate: 5.91% vs 6.28% | Mixed |
Closest papers by meaning
- Recurrent Hierarchical Topic-Guided RNN for Language Generation (ICML 2020 · similarity 0.91)
- textTOvec: DEEP CONTEXTUALIZED NEURAL AUTOREGRESSIVE TOPIC MODELS OF LANGUAGE WITH DISTRIBUTED COMPOSITIONAL PRIOR (ICLR 2019 · similarity 0.89)
- Topic Modeling Revisited: A Document Graph-based Neural Network Perspective (NeurIPS 2021 · similarity 0.88)
- Discovering Discrete Latent Topics with Neural Variational Inference (ICML 2017 · similarity 0.88)
- Modeling Local Dependence in Natural Language with Multi-Channel Recurrent Neural Networks (AAAI 2019 · similarity 0.88)
- LightRNN: Memory and Computation-Efficient Recurrent Neural Networks (NeurIPS 2016 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.