Research map / Architectures and efficiency / RNNs and normalization
Cells in Multidimensional Recurrent Neural Networks
Gundram Leifert, Tobias Strauß, Tobias Grüning, Welf Wustlich, Roger Labahn
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace LSTM with LeakyLP in the lowest 2D layer | MDRNN · IFN/ENIT validation | median label error rate (%): 10,58% -> 8,93% | Improved |
| replace LSTM with Leaky cell in the lowest 2D layer | MDRNN · IFN/ENIT validation | median label error rate (%): 10,58% -> 9,10% | Improved |
| replace LSTM with Stable cell in the lowest 2D layer | MDRNN · IFN/ENIT validation | median label error rate (%): 10,58% -> 9,55% | Improved |
| replace LSTM with LeakyLP in the lowest layer, spread across 10 inits | MDRNN · IFN/ENIT validation | max label error rate (%): max 14,73% -> 9,40% | Improved |
| also use LeakyLP in 2D layer 2 (LeakyLP/LeakyLP/LSTM) (ablation) | MDRNN · IFN/ENIT validation | median label error rate (%): 8,93% -> 8,91% | No clear effect |
| also use LeakyLP in 2D layer 3 (all LeakyLP) (ablation) | MDRNN · IFN/ENIT validation | median label error rate (%): 8,91% -> 9,74% | Hurt |
| raise learning rate from 1e-4 to 1e-3 for LSTM cells (ablation) | LSTM MDRNN · IFN/ENIT validation | median label error rate (%): 10,58% -> 11,44%; max 14,73% -> 30,20% | Hurt |
| raise learning rate from 1e-4 to 5e-4 for LeakyLP cells (ablation) | LeakyLP MDRNN · IFN/ENIT validation | median label error rate (%): 9,74% -> 8,78% | Improved |
| replace LSTM with LeakyLP in the lowest 2D layer | MDRNN · Rimes validation | median label error rate (%): 16,50% -> 15,30% | Improved |
| replace LSTM with Stable cell in the lowest 2D layer | MDRNN · Rimes validation | median label error rate (%): 16,50% -> 15,11% | Improved |
| replace LSTM with Leaky cell in the lowest 2D layer | MDRNN · Rimes validation | median label error rate (%): 16,50% -> 15,85% | Improved |
| also use LeakyLP in 2D layer 2 (LeakyLP/LeakyLP/LSTM) (ablation) | MDRNN · Rimes validation | median label error rate (%): 15,30% -> 14,92% | Improved |
Closest papers by meaning
- Distributed Sequence Memory of Multidimensional Inputs in Recurrent Networks (JMLR 2017 · similarity 0.87)
- Parallelizing Linear Recurrent Neural Nets Over Sequence Length (ICLR 2018 · similarity 0.85)
- Gating Revisited: Deep Multi-layer RNNs That Can Be Trained (TPAMI 2021 · similarity 0.85)
- Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition (NeurIPS 2016 · similarity 0.85)
- Do RNN and LSTM have Long Memory? (ICML 2020 · similarity 0.85)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
