Research map / Architectures and efficiency / Attention and transformers
Stabilizing Transformers for Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace canonical transformer blocks with reordered layer normalization and GRU-type gating | Transformer-XL RL agent · DMLab-30 | Mean Human Norm.: 5.0 ± 0.2 -> 117.6 ± 0.3 | Helped |
| replace LSTM with GTrXL (GRU) | DMLab-30 · DMLab-30 | Mean Human Norm.: 99.3 ± 1.0 -> 117.6 ± 0.3 | Helped |
| replace MERLIN with GTrXL (GRU) | DMLab-30 · DMLab-30 | Mean Human Norm.: 115.2 -> 117.6 ± 0.3 | Helped |
| add identity-map layer-normalization reordering (ablation) | TrXL · DMLab-30 | Mean Human Norm.: 5.0 ± 0.2 -> 107.0 ± 1.2 | Helped |
| use GRU-type gating instead of output gating (ablation) | GTrXL · DMLab-30 | Mean Human Norm.: 112.8 ± 0.8 -> 117.6 ± 0.3 | Helped |
| use GRU-type gating instead of input gating (ablation) | GTrXL · DMLab-30 | Mean Human Norm.: 51.2 ± 13.2 -> 117.6 ± 0.3 | Helped |
| use GRU-type gating instead of highway gating (ablation) | GTrXL · DMLab-30 | Mean Human Norm.: 90.9 ± 12.9 -> 117.6 ± 0.3 | Helped |
| use GRU-type gating instead of sigmoid-tanh gating (ablation) | GTrXL · DMLab-30 | Mean Human Norm.: 101.0 ± 1.3 -> 117.6 ± 0.3 | Helped |
| reduce model capacity by halving attention heads and embedding dimension while retaining GRU gating (ablation) | GTrXL (GRU) · DMLab-30 | Mean Human Norm.: 111.5 ± 0.6 | No clear effect |
| initialize the GRU gate bias to 2 rather than 0 (ablation) | GTrXL (GRU) · Memory Maze | mean return | Helped |
| use gated identity initialization (ablation) | GTrXL (GRU) · Memory Maze | optimization stability | Helped |
| use GRU-type gating rather than an LSTM | Memory Maze · Memory Maze | learning speed | Helped |
Closest papers by meaning
- Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement (NeurIPS 2026 · similarity 0.87)
- Rethinking Transformers in Solving POMDPs (ICML 2024 · similarity 0.87)
- Transformer-Squared: Self-adaptive LLMs (ICLR 2025 · similarity 0.87)
- On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.86)
- Beyond Attention: Breaking the Limits of Transformer Context Length with Recurrent Memory (AAAI 2024 · similarity 0.86)
- AREAL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.