Research map / Architectures and efficiency / RNNs and normalization
Variable Computation in Recurrent Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive partial hidden-state updates | VCRNN-500 versus Elman RNN · Irish traditional tunes test | perplexity: 8.70 -> 8.51 | Helped |
| Use adaptive partial hidden-state updates | VCRNN-1000 versus Elman RNN · Penn Treebank bit-level | bpb: 0.227 -> 0.215 | Helped |
| Use adaptive partial hidden-state updates | VCRNN-1024 versus Elman RNN · Penn Treebank character-level | bpc: 1.47 -> 1.46 | Helped |
| Use adaptive partial hidden-state updates | VCGRU-1024 versus GRU · Penn Treebank character-level | bpc: 1.42 -> 1.42 | No clear effect |
| Use adaptive computation with target m 0.4 (ablation) | VCGRU-1024 · Text8 character-level | bpc: 1.69 -> 1.64 | Helped |
| Use adaptive computation with target m 0.5 (ablation) | VCGRU-1024 · Text8 character-level | bpc: 1.58 -> 1.63 | Helped |
Closest papers by meaning
- Capacity and Trainability in Recurrent Neural Networks (ICLR 2017 · similarity 0.90)
- Fast-Slow Recurrent Neural Networks (NeurIPS 2017 · similarity 0.90)
- Training Recurrent Neural Networks Online by Learning Explicit State Variables (ICLR 2020 · similarity 0.89)
- Efficient recurrent architectures through activity sparsity and sparse back-propagation through time (ICLR 2023 · similarity 0.89)
- Stable Recurrent Models (ICLR 2019 · similarity 0.88)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.