Research map / Bandits and online learning / Online convex optimization and games
A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks
Owen Marschall, Kyunghyun Cho, Cristina Savin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use stochastic rank-1 approximations (UORO, R-KF-RTRL) instead of deterministic ones | online RNN training · Add and Mimic tasks | mean gradient alignment with RTRL: UORO 0.043 on Add and 0.084 on Mimic; R-KF-RTRL 0.050 on Add and 0.092 on Mimic | Hurt |
| use KF-RTRL instead of exact RTRL | online RNN training · Add task | training loss | No clear effect |
| learn A and alpha in KeRNL instead of fixed RFLO feedback | RFLO · Add task | training loss | Hurt |
| initialize recurrent W as orthogonal or Gaussian rather than symmetric or diagonal (ablation) | RFLO · Add task | ability to learn | Improved |
Closest papers by meaning
- Training Recurrent Neural Networks Online by Learning Explicit State Variables (ICLR 2020 · similarity 0.90)
- Online learning of long-range dependencies (NeurIPS 2023 · similarity 0.89)
- Efficient Training of Minimal and Maximal Low-Rank Recurrent Neural Networks (NeurIPS 2025 · similarity 0.88)
- Stable Recurrent Models (ICLR 2019 · similarity 0.88)
- State-Regularized Recurrent Neural Networks (ICML 2019 · similarity 0.88)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
