Research map / Architectures and efficiency / RNNs and normalization
Independently Recurrent Neural Network (IndRNN): Building a Longer and Deeper RNN
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a six-layer IndRNN instead of the strongest listed baseline | Sequential MNIST classification · MNIST | error rate (%): 1.8 -> 1.0 | Helped |
| Use a six-layer IndRNN instead of the strongest listed baseline | Permuted MNIST classification · pMNIST | error rate (%): 4.1 -> 4.0 | Helped |
| Increase the sequence length from 50 to 150 steps (ablation) | IndRNN (6 layers) · PTB-c | BPC: 1.26 -> 1.23 | Helped |
| Use a 21-layer residual IndRNN instead of a 6-layer IndRNN (ablation) | PTB-c with 50-step sequences · PTB-c | BPC: 1.26 -> 1.21 | Helped |
| Increase the IndRNN depth from 4 to 6 layers (ablation) | NTU RGB+D action recognition · NTU RGB+D CS | accuracy: 78.58% -> 81.80% | Helped |
| Increase the IndRNN depth from 4 to 6 layers (ablation) | NTU RGB+D action recognition · NTU RGB+D CV | accuracy: 83.75% -> 87.97% | Helped |
| Use one neuron in the second IndRNN layer instead of the full layer (ablation) | IndRNN on the adding problem · adding problem (T=5000) | MSE | No clear effect |
| Use a residual IndRNN (11 layers) rather than the strongest listed baseline | Word-level Penn Treebank language modeling · PTB-word test | perplexity: 62.4 -> 65.3 | Hurt |
Closest papers by meaning
- Dilated Recurrent Neural Networks (NeurIPS 2017 · similarity 0.89)
- LightRNN: Memory and Computation-Efficient Recurrent Neural Networks (NeurIPS 2016 · similarity 0.88)
- Quasi-Recurrent Neural Networks (ICLR 2017 · similarity 0.88)
- Parallelizing Linear Recurrent Neural Nets Over Sequence Length (ICLR 2018 · similarity 0.88)
- Training biologically plausible recurrent neural networks on cognitive tasks with long-term dependencies (NeurIPS 2023 · similarity 0.88)
- Hierarchically Gated Recurrent Neural Network for Sequence Modeling (NeurIPS 2023 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.