Research map / Reinforcement learning / Policy gradient and actor-critic
Doubly Robust Augmented Transfer for Meta-Reinforcement Learning
Yuankun Jiang, Nuowen Kan, Chenglin Li, Wenrui Dai, Junni Zou, Hongkai Xiong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DRaT doubly robust augmented transfer on top of PEARL | PEARL · Sparse-reward MuJoCo (Ant, Humanoid, Point-Robot with varying dynamics and goals) | average return | Improved |
| add hindsight task relabeling (HTR) to PEARL | PEARL · Ant-Params-Sparse | average return | Hurt |
| add hindsight task relabeling (HTR) to PEARL | PEARL · Humanoid-Params-Sparse | average return | No clear effect |
| add hindsight foresight relabeling (HFR) to PEARL | PEARL · Ant and Humanoid sparse-reward | average return | Improved |
| add approximate inverse RL relabeling (AIR) to PEARL | PEARL · Sparse-reward MuJoCo tasks | average return | Hurt |
| replace hindsight relabeling transfer with DRaE-based value estimation, using the same AIR relabeling strategy | AIR-based transfer · Sparse-reward MuJoCo tasks | asymptotic return and learning speed | Improved |
| add DRaT to PEARL on dense-reward varying-dynamics tasks | PEARL · Walker-2D-Params, Hopper-Params, Point-Robot-Params | average return | Improved |
| use doubly robust augmented estimator instead of plain importance sampling in transfer (IST vs DRaT) (ablation) | DRaT · Dense-reward varying-dynamics tasks | average return | Improved |
| add DRaT versus MQL importance-weighted transfer | MQL · Walker-2D-Params, Hopper-Params | early training speed | Improved |
| use direct unaugmented DR estimator for transferred samples | Point-Robot · Point-Robot, SD of value estimate | standard deviation of value estimate | Hurt |
| add DRaE value estimate in place of the value network alone | PEARL value network V_theta · Point-Robot-Params, Ant-Params-Sparse, Humanoid-Params-Sparse | value estimation error | Improved |
| add DRaT variance control | DRaT · Dense-reward tasks | std of return across seeds | Improved |
Closest papers by meaning
- Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Tasks (AAAI 2022 · similarity 0.91)
- MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components (AAAI 2024 · similarity 0.91)
- MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning (ICLR 2024 · similarity 0.88)
- Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skills (AAAI 2024 · similarity 0.88)
- Meta-Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning (ICML 2024 · similarity 0.88)
- Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
