AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Doubly Robust Augmented Transfer for Meta-Reinforcement Learning

Yuankun Jiang, Nuowen Kan, Chenglin Li, Wenrui Dai, Junni Zou, Hongkai Xiong

NeurIPS 2023 · Reinforcement learning › Policy gradient and actor-critic · approach: Confounding-robust policy evaluation · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add DRaT doubly robust augmented transfer on top of PEARLPEARL · Sparse-reward MuJoCo (Ant, Humanoid, Point-Robot with varying dynamics and goals)average returnImproved
add hindsight task relabeling (HTR) to PEARLPEARL · Ant-Params-Sparseaverage returnHurt
add hindsight task relabeling (HTR) to PEARLPEARL · Humanoid-Params-Sparseaverage returnNo clear effect
add hindsight foresight relabeling (HFR) to PEARLPEARL · Ant and Humanoid sparse-rewardaverage returnImproved
add approximate inverse RL relabeling (AIR) to PEARLPEARL · Sparse-reward MuJoCo tasksaverage returnHurt
replace hindsight relabeling transfer with DRaE-based value estimation, using the same AIR relabeling strategyAIR-based transfer · Sparse-reward MuJoCo tasksasymptotic return and learning speedImproved
add DRaT to PEARL on dense-reward varying-dynamics tasksPEARL · Walker-2D-Params, Hopper-Params, Point-Robot-Paramsaverage returnImproved
use doubly robust augmented estimator instead of plain importance sampling in transfer (IST vs DRaT) (ablation)DRaT · Dense-reward varying-dynamics tasksaverage returnImproved
add DRaT versus MQL importance-weighted transferMQL · Walker-2D-Params, Hopper-Paramsearly training speedImproved
use direct unaugmented DR estimator for transferred samplesPoint-Robot · Point-Robot, SD of value estimatestandard deviation of value estimateHurt
add DRaE value estimate in place of the value network alonePEARL value network V_theta · Point-Robot-Params, Ant-Params-Sparse, Humanoid-Params-Sparsevalue estimation errorImproved
add DRaT variance controlDRaT · Dense-reward tasksstd of return across seedsImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy