Research map / Continual learning / Online and replay-based
Policy Consolidation for Continual Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use policy consolidation instead of PPO baselines | Continuous-control agents · Alternating task pairs | mean reward | Helped |
| Use policy consolidation instead of PPO baselines | Continuous-control agents · Single tasks | mean reward | Helped |
| Use policy consolidation instead of PPO baselines | RoboSumo-Ant-vs-Ant-v0 self-play agents · RoboSumo-Ant-vs-Ant-v0 | win score against baselines | Helped |
| Use policy consolidation rather than clipped PPO | RoboSumo-Ant-vs-Ant-v0 self-play agents · RoboSumo-Ant-vs-Ant-v0, final agent against historical selves | win score | Helped |
| Use policy consolidation rather than fixed-KL PPO with β=2.0 or 5.0 | RoboSumo-Ant-vs-Ant-v0 self-play agents · RoboSumo-Ant-vs-Ant-v0 | win score against baselines | Helped |
| Use reverse KL constraint D_KL(π_k || π_k_old) instead of the original PPO KL direction (ablation) | Policy consolidation · Alternating HalfCheetah and Humanoid tasks | continual-learning performance | Helped |
| Reduce cascade length from 8 policies to 4 or 2 while maintaining the deepest policy's maximum timescale (ablation) | Policy consolidation · Alternating Humanoid tasks | continual-learning performance | Hurt |
| Reduce ω from 4 to 2 or √2 with 8 policies, lowering the maximum timescale (ablation) | Policy consolidation · Alternating Humanoid tasks | continual-learning performance | Hurt |
| Switch tasks at a much faster schedule (ablation) | Policy consolidation · Alternating Humanoid tasks | ability to switch between tasks | Hurt |
| Test hidden cascade policies rather than only the visible policy (ablation) | Policy consolidation · Alternating HumanoidSmallLeg-v0 and HumanoidBigLeg-v0; evaluated on HumanoidSmallLeg-v0 | reward | Hurt |
Closest papers by meaning
- Self-Composing Policies for Scalable Continual Reinforcement Learning (ICML 2024 · similarity 0.90)
- A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control (ICML 2025 · similarity 0.86)
- Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning (NeurIPS 2026 · similarity 0.86)
- Continual Reinforcement Learning by Planning with Online World Models (ICML 2025 · similarity 0.86)
- Knowledge Retention in Continual Model-Based Reinforcement Learning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.