Research map / Reinforcement learning / Offline RL
Offline Meta-Reinforcement Learning with Online Self-Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add self-supervised online meta-training with rewards generated from the offline-trained reward decoder | SMAC · Six meta-RL environments | post-adaptation return | Helped |
| Add self-supervised online meta-training | SMAC · BOReL comparisons across six environments | post-adaptation return | Helped |
| Add self-supervised online meta-training | SMAC · MACAW comparisons across six environments | post-adaptation return | Helped |
| Use the AWAC actor update instead of the PEARL actor update (ablation) | SMAC · Sawyer Manipulation | post-adaptation return | Helped |
| Replace the AWAC actor update with the PEARL actor loss (ablation) | SMAC · Sawyer Manipulation | post-adaptation return | Hurt |
| Replace AWAC with SAC as the underlying offline RL algorithm (ablation) | SMAC · More difficult meta-RL tasks | post-adaptation return | Hurt |
| Train the encoder using only the Q-loss in both offline and online phases, rather than the reward loss (ablation) | SMAC · Walker Param, Hopper Param, and Humanoid | post-adaptation return | Hurt |
| Train the encoder with reward and Q-loss offline, then only Q-loss online (ablation) | SMAC · Walker Param, Hopper Param, and Humanoid | post-adaptation return | No clear effect |
| Add the PEARL actor loss to the AWAC actor loss during self-supervised training (ablation) | SMAC · Six meta-RL environments | post-adaptation return | Mixed |
| Replace the actor update with behavior cloning (ablation) | SMAC · Six meta-RL environments | post-adaptation return | Hurt |
| Use self-supervised interaction on test tasks instead of meta-training tasks (ablation) | SMAC · Sawyer Manipulation with eight possible behaviors | post-adaptation test return | Helped |
| Use self-supervised interaction on meta-training tasks instead of test tasks (ablation) | SMAC · Sawyer Manipulation with eight possible behaviors | post-adaptation test return | No clear effect |
Closest papers by meaning
- Offline Meta-Reinforcement Learning with Advantage Weighting (ICML 2021 · similarity 0.95)
- Model-Based Offline Meta-Reinforcement Learning with Regularization (ICLR 2022 · similarity 0.93)
- MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components (AAAI 2024 · similarity 0.91)
- Enhancing Online Reinforcement Learning with Meta-Learned Objective from Offline Data (AAAI 2025 · similarity 0.91)
- Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skills (AAAI 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.