Research map / Architectures and efficiency / RNNs and normalization
Parallelizing Legendre Memory Unit Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the original LMU with the simplified DN-based model | LMU · psMNIST test | accuracy: 97.15 -> 98.49 | Helped |
| Use the simplified DN-based model instead of the LMU | LMU · Mackey-Glass test | NRMSE: 0.049 -> 0.044 | Helped |
| Use the DN-only model instead of the LSTM | LSTM · IMDB | accuracy: 87.29 -> 89.10 | Helped |
| Use the DN-only model instead of the LSTM | LSTM · QQP 390k/8k/8k | accuracy: 82.58 -> 86.95 | Helped |
| Use the DN-only model instead of the LSTM | LSTM · QQP 280k/80k/40k | accuracy: 81.4 -> 85.36 | Helped |
| Use the DN-only model instead of the LSTM | LSTM · SNLI | accuracy: 77.6 -> 78.85 | Helped |
| Pre-train on Amazon Reviews and fine-tune for sentiment classification | Our Model · IMDB | accuracy: 92.88 -> 93.20 | Helped |
| Use the model instead of the LSTM | LSTM · text8 test | bits per character: 1.65 -> 1.61 | Helped |
| Use the model instead of the LSTM | LSTM · IWSLT'15 En-Vi TED tst2013 | BLEU case sensitive: 23.3 -> 25.5^a | Helped |
| Parallelize training of the DN instead of using its recurrent LTI implementation (ablation) | Our Model · psMNIST and Mackey-Glass | training speedup: 220x and 64x | Helped |
| Use the parallel implementation rather than the original LMU (ablation) | LMU · Mackey-Glass | training speedup: about 200x | Helped |
| Add a self-attention layer after the final block (ablation) | text8 language model · text8 test | generalization | Helped |
Closest papers by meaning
- Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks (NeurIPS 2019 · similarity 0.89)
- Parallelizing Linear Recurrent Neural Nets Over Sequence Length (ICLR 2018 · similarity 0.87)
- LMUFormer: Low Complexity Yet Powerful Spiking Model With Legendre Memory Units (ICLR 2024 · similarity 0.86)
- Resurrecting Recurrent Neural Networks for Long Sequences (ICML 2023 · similarity 0.86)
- Parallelizing non-linear sequential models over the sequence length (ICLR 2024 · similarity 0.84)
- Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.