Research map / Architectures and efficiency / RNNs and normalization
Quasi-Recurrent Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the densely-connected 4-layer LSTM with a densely-connected 4-layer QRNN | IMDb sentiment classifier · IMDb test | accuracy: 90.9 -> 91.4 | Helped |
| replace the densely-connected 4-layer LSTM with a densely-connected 4-layer QRNN | IMDb sentiment classifier · IMDb train | time per epoch: 480 -> 150 | Helped |
| use k=4 in the first layer instead of the default k=2 (ablation) | Densely-connected 4-layer QRNN · IMDb test | accuracy: 91.4 -> 91.1 | Hurt |
| use k=4 in the first layer instead of the default k=2 (ablation) | Densely-connected 4-layer QRNN · IMDb train | time per epoch: 150 -> 160 | Hurt |
| replace the medium LSTM with a medium QRNN | Penn Treebank language model · Penn Treebank test | perplexity: 82.0 -> 79.9 | Helped |
| add zoneout (p=0.1) to the QRNN pooling layer (ablation) | Medium QRNN · Penn Treebank validation | perplexity: 82.9 -> 82.1 | Helped |
| add zoneout (p=0.1) to the QRNN pooling layer (ablation) | Medium QRNN · Penn Treebank test | perplexity: 79.9 -> 78.3 | Helped |
| replace a character-level 4-layer LSTM with a character-level 4-layer QRNN | IWSLT German-English translation model · IWSLT German-English TED.tst2014 | BLEU: 16.53 -> 19.41 | Helped |
| replace a character-level 4-layer LSTM with a character-level 4-layer QRNN | IWSLT German-English translation model · IWSLT German-English training | train time per epoch: 4.2 hrs/epoch -> 1.0 hrs/epoch | Helped |
Closest papers by meaning
- Parallelizing Linear Recurrent Neural Nets Over Sequence Length (ICLR 2018 · similarity 0.90)
- FlashRNN: Unlocking Parallel Training of Nonlinear RNNs for Large Language Models (ICLR 2026 · similarity 0.89)
- Recurrent Stacking of Layers for Compact Neural Machine Translation Models (AAAI 2019 · similarity 0.89)
- LightRNN: Memory and Computation-Efficient Recurrent Neural Networks (NeurIPS 2016 · similarity 0.89)
- Recurrently Controlled Recurrent Networks (NeurIPS 2018 · similarity 0.88)
- Parallelizing non-linear sequential models over the sequence length (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.