Research map / Reinforcement learning / Offline RL
Offline Meta-Reinforcement Learning with Advantage Weighting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the advantage-regression auxiliary loss to the inner-loop AWR policy update (ablation) | MACAW policy adaptation · Cheetah-Velocity | learning speed and asymptotic performance | Helped |
| Add weight transform layers instead of standard fully connected layers (ablation) | MACAW policy and value networks · Cheetah-Velocity | learning speed and stability | Helped |
| Add weight transform layers instead of standard fully connected layers (ablation) | MACAW · Half-Cheetah Direction | training speed and stability | Helped |
| Add weight transform layers instead of standard fully connected layers (ablation) | MACAW · MuJoCo tasks other than Half-Cheetah Direction | final performance | Helped |
| Replace weight transform layers with equal-width fully connected layers (ablation) | MACAW · Cheetah-Velocity | learning speed and stability | Hurt |
| Replace weight transform layers with wider fully connected layers at roughly equal parameter count (ablation) | MACAW · Cheetah-Velocity | learning speed | Hurt |
| Use near-optimal rather than weaker data for inner-loop adaptation (ablation) | MACAW without the enriched policy update · Cheetah-Velocity | post-adaptation performance | Helped |
| Use weaker, pre-convergence data rather than near-optimal data for inner-loop adaptation (ablation) | MACAW without the enriched policy update · Cheetah-Velocity | post-adaptation performance | Hurt |
| Use 20k online fine-tuning steps after offline adaptation | MACAW · Cheetah-Vel OOD | reward improvement: 44.0 (14.5) | Helped |
| Use 20k online fine-tuning steps after offline adaptation | Offline PEARL+FT · Cheetah-Vel OOD | reward improvement: -11.5 (3.5) | Hurt |
| Increase offline PEARL training-task count beyond the middle regime (ablation) | Offline PEARL · Cheetah-Velocity task-sparsity evaluation | test performance | Mixed |
| Train using only three meta-training tasks (ablation) | MACAW · Cheetah-Velocity task-sparsity evaluation | test performance | No clear effect |
Closest papers by meaning
- Offline Meta-Reinforcement Learning with Online Self-Supervision (ICML 2022 · similarity 0.95)
- Model-Based Offline Meta-Reinforcement Learning with Regularization (ICLR 2022 · similarity 0.94)
- FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularization (ICLR 2021 · similarity 0.93)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.92)
- Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-Based Offline Meta Reinforcement Learning (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.