Research map / Reinforcement learning / Offline RL
Representation Matters: Offline Pretraining for Sequential Decision Making
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain state representations with unsupervised objectives | Downstream imitation learning · Gym-MuJoCo imitation tasks | normalized average reward: 1.5x | Helped |
| Pretrain state representations with unsupervised objectives | Downstream offline RL · Gym-MuJoCo offline RL tasks | normalized average reward: 2.5x | Helped |
| Pretrain state representations with unsupervised objectives | Downstream online RL · Partially observable Gym-MuJoCo online RL tasks | normalized average reward: 15% | Helped |
| Use a longer sub-trajectory (k+1=8 instead of k+1=2) | Contrastive self-prediction objectives · Gym-MuJoCo downstream tasks | average reward | Helped |
| Use raw-state forward prediction rather than contrastive self-prediction | Representation pretraining · Gym-MuJoCo downstream tasks | average reward | Hurt |
| Predict future representations with DeepMDP or bisimulation objectives | Representation pretraining · Gym-MuJoCo downstream tasks | average reward | Hurt |
| Add action reconstruction or prediction, or include actions in the input sequence (ablation) | ACL for imitation learning · Gym-MuJoCo imitation tasks | average reward | Hurt |
| Add reward reconstruction or prediction, or include rewards in the input sequence (ablation) | ACL for imitation learning · Gym-MuJoCo imitation tasks | average reward | Hurt |
| Add action reconstruction or prediction, or include actions in the input sequence (ablation) | ACL for offline RL · Gym-MuJoCo offline RL tasks | average reward | Helped |
| Add reward reconstruction or prediction, or include rewards in the input sequence (ablation) | ACL for offline RL · Gym-MuJoCo offline RL tasks | average reward | Helped |
| Add action reconstruction or prediction, or include actions in the input sequence (ablation) | ACL for online RL · Partially observable Gym-MuJoCo online RL tasks | average reward | Helped |
| Add reward reconstruction or prediction, or include rewards in the input sequence (ablation) | ACL for online RL · Partially observable Gym-MuJoCo online RL tasks | average reward | Helped |
Closest papers by meaning
- Reining Generalization in Offline Reinforcement Learning via Representation Distinction (NeurIPS 2023 · similarity 0.93)
- Behavior Prior Representation learning for Offline Reinforcement Learning (ICLR 2023 · similarity 0.93)
- A Policy-Guided Imitation Approach for Offline Reinforcement Learning (NeurIPS 2022 · similarity 0.92)
- Leveraging Offline Data in Online Reinforcement Learning (ICML 2023 · similarity 0.92)
- Is Value Learning Really the Main Bottleneck in Offline RL? (NeurIPS 2024 · similarity 0.92)
- Foundation Policies with Hilbert Representations (ICML 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.