Research map / Architectures and efficiency / RNNs and normalization
Long Short-Term Memory with Dynamic Skip Connections
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add dynamic skip connections | LSTM · CoNLL2003 test | F1: 91.21 -> 91.56 | Helped |
| Add dynamic skip connections | Regularized LSTM · Penn Treebank test | perplexity: 82.7 to 78.5 | Helped |
| Add dynamic skip connections | CharLM · Penn Treebank test | perplexity: 78.9 to 76.5 | Helped |
| Replace the standard recurrent connection with dynamic skips | LSTM · IMDB test | accuracy: 89.1 -> 90.1 | Helped |
| Add dynamic skip connections with λ=0.5 and K=10 | LSTM · Number prediction, sequence length 11 | accuracy: 70.4 -> 90.5 | Helped |
| Add dynamic skip connections with λ=0.5 and K=10 | LSTM · Number prediction, sequence length 21 | accuracy: 26.4 -> 88.5 | Helped |
| Use dynamic skips instead of attention-based connections | LSTM · CoNLL2003 test | F1: 91.23 -> 91.56 | Helped |
| Use dynamic skips instead of fixed skip length 3 | LSTM · CoNLL2003 test | F1: 91.14 -> 91.56 | Helped |
| Use dynamic skips instead of fixed skip length 5 | LSTM · CoNLL2003 test | F1: 91.16 -> 91.56 | Helped |
| Set λ to 0.5 rather than 1 (ablation) | LSTM with dynamic skip, K=10 · Number prediction, sequence length 11 test | accuracy: 80.5 -> 90.5 | Helped |
| Set λ to 0.5 rather than 1 (ablation) | LSTM with dynamic skip, K=10 · Number prediction, sequence length 21 test | accuracy: 77.7 -> 88.5 | Helped |
| Use dynamic skips instead of attention-based connections | LSTM · IMDB test | accuracy: 89.4 -> 90.1 | Helped |
Closest papers by meaning
- Towards Non-Saturating Recurrent Units for Modelling Long-Term Dependencies (AAAI 2019 · similarity 0.87)
- Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks (ICLR 2018 · similarity 0.86)
- Mogrifier LSTM (ICLR 2020 · similarity 0.86)
- Explain My Surprise: Learning Efficient Long-Term Memory by predicting uncertain outcomes (NeurIPS 2022 · similarity 0.86)
- xLSTM: Extended Long Short-Term Memory (NeurIPS 2024 · similarity 0.85)
- Memorization Sinks: Isolating Memorization during LLM Training (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.