Research map / Reinforcement learning / Visual and representation RL
Hindsight Foresight Relabeling for Meta-Reinforcement Learning
Michael Wan, Jian Peng, Tanmay Gangwani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Relabel trajectories using post-adaptation utility rather than trajectory return | PEARL meta-training on sparse-reward tasks · Sparse-reward MuJoCo manipulation and locomotion tasks | average return or success rate | Improved |
| Relabel trajectories using post-adaptation utility | PEARL meta-training · Cheetah-Highdim | average return | Improved |
| Add HFR trajectory relabeling | PEARL meta-training · Cheetah-Vel and Ant-Vel | average return | No clear effect |
| Add HFR trajectory relabeling | PEARL meta-training · Tasks with differing transition dynamics and reward functions | average return | Improved |
| Use smaller utility-estimation batch sizes, especially 16 or 32 (ablation) | HFR · MuJoCo environments evaluated in the batch-size ablation | average return | Hurt |
| Remove the log-partition normalization (ablation) | HFR relabeling distribution · MuJoCo environments evaluated in the partition-function ablation | average return | Hurt |
| Replace the true reward function with a learned reward function for relabeling (ablation) | HFR · MuJoCo environments evaluated in the reward-function ablation | average return | No clear effect |
| Replace stochastic soft-max relabeling with deterministic hard-max relabeling (ablation) | HFR · MuJoCo environments evaluated in the soft-max versus hard-max ablation | average return | Hurt |
| Use negative Bellman error instead of post-adaptation return as the trajectory utility (ablation) | HFR (HFR-Bellman) · Environments shown in the HFR-Bellman comparison | average return | Hurt |
| Relabel trajectories to tasks selected using post-adaptation utility | PEARL context embeddings · Four-Corners | task-classification accuracy: 76.98% -> 89.65% | Improved |
| Relabel trajectories to tasks selected using post-adaptation utility | PEARL context embeddings · Cheetah-Vel-Sparse | task-classification accuracy: 57.58% -> 74.33% | Improved |
Closest papers by meaning
- Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RL (NeurIPS 2021 · similarity 0.94)
- PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling (ICML 2024 · similarity 0.88)
- MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components (AAAI 2024 · similarity 0.88)
- DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- How Does Goal Relabeling Improve Sample Efficiency? (ICML 2024 · similarity 0.87)
- AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
