AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Recurrent Orthogonal Networks and Long-Memory Tasks

Mikael Henaff, Arthur Szlam, Yann LeCun

ICML 2016 · Architectures and efficiency › RNNs and normalization · approach: RNN training dynamics and gradients · 55 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use random orthogonal rather than identity transition initializationLT-RNN · Fixed-length copy tasktask performanceImproved
Use identity rather than random orthogonal transition initializationLT-RNN · Adding tasktask performanceImproved
Use random orthogonal rather than identity transition initializationLT-RNN · Adding tasktask performanceHurt
Use identity rather than random orthogonal transition initializationLT-RNN · Fixed-length copy tasktask performanceHurt
Add l2 pooling to the decoderLT-ORNN · Adding tasktask performanceImproved
Add l2 pooling to the decoderLT-ORNN · Fixed-length copy tasktask performanceNo clear effect
Use an LT-RNN architecture, with identity, random orthogonal, or pooled random orthogonal initializationLT-RNN · Variable length copy task (T=100)task performanceHurt
Use an LSTM rather than an LT-RNNVariable length copy task (T=100) · Variable length copy tasktask performanceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy