Research map / Reinforcement learning / Offline RL
Offline RL Policies Should Be Trained to be Adaptive
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train an uncertainty-adaptive policy that updates its value-ensemble belief during evaluation | APE-V on Procgen Mazes with 200 train levels · Procgen Mazes, 200 train levels | test success rate: 0.23 { $\pm 0.02} -> 0.31 { $\pm 0.04} | Helped |
| train an uncertainty-adaptive policy that updates its value-ensemble belief during evaluation | APE-V on Procgen Mazes with 1000 train levels · Procgen Mazes, 1000 train levels | test success rate: 0.75 { $\pm 0.04} -> 0.79 { $\pm 0.04} | Helped |
| remove the ensemble and uncertainty adaptation (ablation) | APE-V on Procgen Mazes with 200 train levels · Procgen Mazes, 200 train levels | test success rate: 0.31 { $\pm 0.04} -> 0.24 { $\pm .02} | Hurt |
| replace adaptive value weighting with conservative LCB values (ablation) | APE-V on Procgen Mazes with 200 train levels · Procgen Mazes, 200 train levels | test success rate: 0.31 { $\pm 0.04} -> 0.23 { $\pm 0.02} | Hurt |
| remove the ensemble and uncertainty adaptation (ablation) | APE-V on Procgen Mazes with 1000 train levels · Procgen Mazes, 1000 train levels | test success rate: 0.79 { $\pm 0.04} -> 0.75 { $\pm 0.03} | Hurt |
| replace adaptive value weighting with conservative LCB values (ablation) | APE-V on Procgen Mazes with 1000 train levels · Procgen Mazes, 1000 train levels | test success rate: 0.75 { $\pm 0.04} -> 0.79 { $\pm 0.04} | No clear effect |
| use APE-V rather than SAC-N | D4RL halfcheetah-medium · halfcheetah-medium | normalized average return: 67.5±1.2 -> 69.1 ± 0.4 | Helped |
| use APE-V rather than SAC-N | D4RL halfcheetah-medium-replay · halfcheetah-medium-replay | normalized average return: 63.9±0.8 -> 64.6 ± 0.9 | Helped |
| use APE-V rather than SAC-N | D4RL hopper-medium-replay · hopper-medium-replay | normalized average return: 101.8±0.5 -> 98.5 ± 0.5 | Hurt |
| use APE-V rather than SAC-N | D4RL walker2d-medium · walker2d-medium | normalized average return: 87.9±0.2 -> 90.3 ± 1.6 | Helped |
| use APE-V rather than SAC-N | D4RL walker2d-medium-replay · walker2d-medium-replay | normalized average return: 78.7±0.7 -> 82.9 ± 0.4 | Helped |
| use APE-V rather than the conservative ensemble baseline | Locked Doors with CIFAR-10 · Locked Doors with CIFAR-10 | success rate | Helped |
Closest papers by meaning
- Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian (ICLR 2023 · similarity 0.93)
- Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees (ICLR 2024 · similarity 0.91)
- From Static Policies to Adaptive Priors in Offline Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
- Selective Uncertainty Propagation in Offline RL (AAAI 2025 · similarity 0.90)
- Bayesian Design Principles for Offline-to-Online Reinforcement Learning (ICML 2024 · similarity 0.90)
- Improving Offline RL by Blending Heuristics (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.