Research map / Reinforcement learning / Policy gradient and actor-critic
Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgetting
Jorge Mendez, Boyu Wang, Eric Eaton
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn each new task's coefficients by policy gradient in the span of a shared factor dictionary | LPG-FTW versus STL · Six MuJoCo domains | learning speed | Improved |
| learn each new task's coefficients by policy gradient in the span of a shared factor dictionary | LPG-FTW versus STL · Five of six MuJoCo domains | final performance | Improved |
| update the shared factor dictionary using the current task's information | LPG-FTW · Six MuJoCo domains | average return after dictionary update | Improved |
| retain a shared factor dictionary and task-specific coefficients rather than train isolated policies | LPG-FTW versus PG-ELLA · Six MuJoCo domains | catastrophic forgetting | Improved |
| use a shared single policy across tasks | EWC · MuJoCo domains | learning speed | Mixed |
| use a shared single policy across tasks | EWC · Two MuJoCo domains | performance relative to STL | Hurt |
| learn each new task's coefficients in the shared factor span | LPG-FTW versus baselines · Meta-World MT10 | learning speed | Improved |
| learn each new task's coefficients in the shared factor span | LPG-FTW versus baselines · Meta-World MT50 | learning speed | Improved |
| use the shared factor dictionary after sequential training | LPG-FTW · Meta-World MT10 | catastrophic forgetting | Mixed |
| use the shared factor dictionary after sequential training | LPG-FTW · Meta-World MT50 | catastrophic forgetting | Improved |
| increase EWC network capacity (ablation) | EWC · Meta-World MT10 and MT50 | learning and forgetting performance | No clear effect |
| initialize dictionary columns with separate single-task solutions instead of incremental error vectors (ablation) | LPG-FTW initialization · Early sequential tasks | learning speed | Hurt |
Closest papers by meaning
- Policy and Value Transfer in Lifelong Reinforcement Learning (ICML 2018 · similarity 0.89)
- Lifelong Hyper-Policy Optimization with Multiple Importance Sampling Regularization (AAAI 2022 · similarity 0.89)
- State Abstractions for Lifelong Reinforcement Learning (ICML 2018 · similarity 0.89)
- Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Learning Optimal Deterministic Policies with Stochastic Policy Gradients (ICML 2024 · similarity 0.87)
- Oracle-Efficient Reinforcement Learning for Max Value Ensembles (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
