Research map / Reinforcement learning / Visual and representation RL
Understanding and Preventing Capacity Loss in Reinforcement Learning
Clare Lyle, Mark Rowland, Will Dabney
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on a sequence of changing target functions instead of a single target | Neural networks on MNIST and value-based RL agents · Non-stationary target-fitting tasks | target-fitting error | Hurt |
| Add Initial Feature Regularization (InFeR) | Rainbow · Atari 57-game benchmark | human-normalized score | Improved |
| Add InFeR | Rainbow · Montezuma's Revenge | performance | Improved |
| Add InFeR | Double DQN · Montezuma's Revenge | performance | Improved |
| Add InFeR | Double DQN · Atari benchmark | average human-normalized score | Improved |
| Add InFeR | Rainbow · Asteroids and Jamesbond | performance | Hurt |
| Add InFeR to the regression objective | Fully connected network · Non-stationary MNIST random reward prediction | target-fitting capacity | Improved |
| Add InFeR | Rainbow · Atari games | feature rank | Improved |
| Add random features to the learned representation instead of using InFeR (ablation) | Rainbow · Atari games | performance | No clear effect |
| Double the width of the penultimate layer while using InFeR (ablation) | Rainbow · Games where InFeR hurt in the original network | performance gap versus Rainbow | Improved |
| Use a random cumulant prediction auxiliary task | DQN · Montezuma's Revenge | performance | Improved |
| Use a random cumulant prediction auxiliary task | DQN · Complex dense-reward games | learning progress | Hurt |
Closest papers by meaning
- On Double Descent in Reinforcement Learning with LSTD and Random Features (ICLR 2024 · similarity 0.87)
- No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO (NeurIPS 2024 · similarity 0.87)
- Parseval Regularization for Continual Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning (ICML 2024 · similarity 0.86)
- Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- Towards Provable Emergence of In-Context Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
