AI Paper Atlas

Research map / Reinforcement learning / Visual and representation RL

Hindsight Foresight Relabeling for Meta-Reinforcement Learning

Michael Wan, Jian Peng, Tanmay Gangwani

ICLR 2022 · Reinforcement learning › Visual and representation RL · approach: Task-inference meta-RL · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Relabel trajectories using post-adaptation utility rather than trajectory returnPEARL meta-training on sparse-reward tasks · Sparse-reward MuJoCo manipulation and locomotion tasksaverage return or success rateImproved
Relabel trajectories using post-adaptation utilityPEARL meta-training · Cheetah-Highdimaverage returnImproved
Add HFR trajectory relabelingPEARL meta-training · Cheetah-Vel and Ant-Velaverage returnNo clear effect
Add HFR trajectory relabelingPEARL meta-training · Tasks with differing transition dynamics and reward functionsaverage returnImproved
Use smaller utility-estimation batch sizes, especially 16 or 32 (ablation)HFR · MuJoCo environments evaluated in the batch-size ablationaverage returnHurt
Remove the log-partition normalization (ablation)HFR relabeling distribution · MuJoCo environments evaluated in the partition-function ablationaverage returnHurt
Replace the true reward function with a learned reward function for relabeling (ablation)HFR · MuJoCo environments evaluated in the reward-function ablationaverage returnNo clear effect
Replace stochastic soft-max relabeling with deterministic hard-max relabeling (ablation)HFR · MuJoCo environments evaluated in the soft-max versus hard-max ablationaverage returnHurt
Use negative Bellman error instead of post-adaptation return as the trajectory utility (ablation)HFR (HFR-Bellman) · Environments shown in the HFR-Bellman comparisonaverage returnHurt
Relabel trajectories to tasks selected using post-adaptation utilityPEARL context embeddings · Four-Cornerstask-classification accuracy: 76.98% -> 89.65%Improved
Relabel trajectories to tasks selected using post-adaptation utilityPEARL context embeddings · Cheetah-Vel-Sparsetask-classification accuracy: 57.58% -> 74.33%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy