AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Do RNN and LSTM have Long Memory?

Jingyu Zhao, Feiqing Huang, Jia Lv, Yanjie Duan, Zhen Qin, Guodong Li, Guangjian Tian

ICML 2020 · Architectures and efficiency › RNNs and normalization · approach: LSTM design and memory · 40 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add the memory filter to RNNRNN · ARFIMA, DJI, Traffic, and TreeRMSE: MRNN: 1.1620 -> 1.0880; 0.2605 -> 0.2487; 336.44 -> 333.72; 0.2871 -> 0.2818Improved
Add the memory filter to LSTM cell statesLSTM · ARFIMA, DJI, Traffic, and TreeRMSE: 1.1340 -> 1.1490Hurt
Use a fixed memory parameter instead of dynamic d (ablation)MRNN · ARFIMA, DJI, Traffic, and TreeRMSE: 1.0880 -> 1.1010; 0.2487 -> 0.2472; 333.72 -> 333.36; 0.2818 -> 0.2822Mixed
Use a fixed memory parameter instead of dynamic d (ablation)MLSTM · ARFIMA, DJI, Traffic, and TreeRMSE: 1.1490 -> 1.1580; 0.2531 -> 0.2540; 337.83 -> 337.78; 0.2859 -> 0.2859Mixed
Increase truncation length K (ablation)MRNN and MRNNF · ARFIMA, DJI, Traffic, and TreeRMSEImproved
Increase truncation length K (ablation)MLSTM and MLSTMF · ARFIMA, DJI, Traffic, and TreeRMSEHurt
Use K=25 rather than the default K=100 (ablation)MLSTM and MLSTMF · ARFIMA and TrafficRMSEImproved
Add a fixed memory filter to RNNRNN · CMU-MOSIMAE: 1.5028 -> 1.4953Improved
Add a fixed memory filter to LSTM cell statesLSTM · CMU-MOSIMAE: 1.4978 -> 1.4972No clear effect
Use MLSTMF50 as the first layer, with LSTM as the secondTwo-layer sentiment classifier · Paper ReviewsAccuracy: 0.3021 -> 0.3110Improved
Use MLSTMF50 as the first layer, with LSTM as the secondTwo-layer sentiment classifier · Paper ReviewsCross-entropy loss: 1.5752 -> 1.4758Improved
Use memory-augmented models instead of RNN on a short-memory synthetic seriesMRNN(F) and MLSTM(F) · RNN synthetic datasetForecast errorNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy