Research map / Reinforcement learning / Policy gradient and actor-critic
Prediction and Control in Continual Reinforcement Learning
Nishanth Anand, Doina Precup
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use permanent and transient value components instead of a single DQN value function | PT-DQN-0.5x · JellyBeanWorld | reward rate | Improved |
| Use permanent and transient value components instead of a single DQN value function | PT-DQN-0.5x · MinAtar | return | Improved |
| Use permanent and transient value components instead of a single Q-learning value function | PT-Q-learning · Tabular gridworld | episodic return | Improved |
| Use permanent and transient value components instead of a single Q-learning value function | PT-Q-learning · Minigrid | return | Improved |
| Use permanent and transient value components instead of a single TD value estimate | PT-TD · Prediction tasks (discrete grid, continuous grid, and Minigrid) | RMSVE on the current task | Improved |
| Use permanent and transient value components instead of a single TD value estimate | PT-TD · Prediction tasks (discrete grid, continuous grid, and Minigrid) | MSE on other tasks | Improved |
| Reset the value-function weights at each task switch instead of preserving learned estimates (ablation) | TD-learning and Q-learning · Semi-continual prediction and control experiments | current-task prediction error / episodic return | Hurt |
| Add a large experience replay buffer to DQN (ablation) | DQN · JellyBeanWorld and MinAtar continual RL experiments | reward rate / return | Hurt |
| Use multiple task-specific Q-value heads with a shared trunk (ablation) | DQN · JellyBeanWorld and MinAtar continual RL experiments | reward rate / return | Hurt |
| Vary the permanent-update interval k and transient decay factor λ (ablation) | PT-Q-learning · Continual tabular gridworld | episodic return | Mixed |
| Set permanent-update frequency far above or below task-switch frequency (ablation) | PT-Q-learning · Continual tabular gridworld | episodic return | No clear effect |
Closest papers by meaning
- Discerning Temporal Difference Learning (AAAI 2024 · similarity 0.89)
- Temporal-Difference Variational Continual Learning (NeurIPS 2025 · similarity 0.88)
- Pausing Policy Learning in Non-stationary Reinforcement Learning (ICML 2024 · similarity 0.88)
- Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy (NeurIPS 2024 · similarity 0.88)
- Continual Reinforcement Learning by Planning with Online World Models (ICML 2025 · similarity 0.88)
- Ergodic Risk Measures: Towards a Risk-Aware Foundation for Continual Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
