Research map / Learning theory and algorithms / Submodular and game algorithms
Optimal Kronecker-Sum Approximation of Real Time Recurrent Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OK instead of KF-RTRL-AVG | RTRL approximation · Penn TreeBank validation | BPC: 1.82 -> 1.74 | Helped |
| Use OK instead of KF-RTRL-AVG | RTRL approximation · Penn TreeBank test | BPC: 1.77 -> 1.69 | Helped |
| Use 8-OK instead of TBPTT-25 | RHN language model · Penn TreeBank validation | BPC: 1.73 -> 1.74 | Hurt |
| Use 8-OK instead of TBPTT-25 | RHN language model · Penn TreeBank test | BPC: 1.69 -> 1.69 | No clear effect |
| Use 16-OK instead of 16-KF-RTRL-AVG | RHN on the Copy task · Copy task | learned sequence length: 32 -> 42 | Helped |
| Use 16-OK instead of TBPTT | RHN on the Copy task · Copy task | learned sequence length: 33 -> 42 | Helped |
| Approximate the diagonal factor D with a rank-16 matrix in KF-RTRL (ablation) | KF-RTRL on the Copy task · Copy task | learned sequence length | No clear effect |
| Use KTP instead of TBPTT | RTRL approximation · Copy task | learned sequence length | Mixed |
Closest papers by meaning
- Approximating Real-Time Recurrent Learning with Random Kronecker Factors (NeurIPS 2018 · similarity 0.96)
- Practical Real Time Recurrent Learning with a Sparse Approximation (ICLR 2021 · similarity 0.90)
- Exploring the Promise and Limits of Real-Time Recurrent Learning (ICLR 2024 · similarity 0.90)
- Kronecker Recurrent Units (ICML 2018 · similarity 0.88)
- Training Recurrent Neural Networks Online by Learning Explicit State Variables (ICLR 2020 · similarity 0.88)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.