Research map / Reinforcement learning / Offline RL
Behavior Prior Representation learning for Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain encoder with BPR behavior-cloning objective (100k steps) then freeze it for offline RL | TD3+BC, CQL, EDAC on D4RL halfcheetah/hopper/walker2d · D4RL MuJoCo expert and medium-expert | convergence speed and policy performance | Helped |
| Use BPR encoder instead of other representation objectives (Fourier, ACL, Contrastive, Inverse, Forward, Super) | BRAC fine-tuning on DMC tasks · DMC 16 tasks | IQM normalized score: 89.99 vs 82.24 (Fourier, next best) | Helped |
| Replace BPR encoder with pure contrastive/forward/inverse objectives | representation objectives for offline RL on DMC · DMC 16 tasks | IQM normalized score: Forward 18.59 and Contrastive 44.87 vs BPR 89.99 | Hurt |
| Pretrain encoder with BPR objective before DrQ+BC on visual DMC | DrQ+BC · V-D4RL cheetah-run-medium-expert (original env) | cumulative return: 45.50 -> 113.64 | Helped |
| Pretrain encoder with BPR objective before DrQ+BC | DrQ+BC · V-D4RL cheetah-run-medium-expert, hard distractions, distractor test | cumulative return: 18.88 -> 16.72 | No clear effect |
| Pretrain encoder with BPR objective before DrQ+BC | DrQ+BC · V-D4RL cheetah-run-medium-expert, easy distractions, distractor train | cumulative return: 425.15 -> 710.11 | Helped |
| BPR objective in visual DMC against DRIML, HOMER, CURL, Inverse, SGI | DrQ+BC visual offline RL · V-D4RL DMC tasks | normalized score | Helped |
| Use BPR pretraining instead of SGI (most time-consuming baseline) | visual representation pretraining · cheetah_run_expert 100K timesteps wall-clock | training time: BPR 2.5x faster than SGI | Helped |
| Pretrain encoder with BPR vs. vanilla CQL (no encoder) and encoder trained with critic loss (ablation) | CQL on Halfcheetah-medium-expert-v2 · effective dimension of value network penultimate layer | effective dimension and performance | Helped |
| Train encoder jointly with critic loss instead of BPR (ablation) | CQL on Halfcheetah-medium-expert-v2 · effective dimension over training | effective dimension | Mixed |
| Uninformative (uniform) behavior policy used for BPR | BPR · offline RL | policy improvement over behavior | No clear effect |
Closest papers by meaning
- Representation Matters: Offline Pretraining for Sequential Decision Making (ICML 2021 · similarity 0.93)
- Reining Generalization in Offline Reinforcement Learning via Representation Distinction (NeurIPS 2023 · similarity 0.91)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
- Offline RL with Observation Histories: Analyzing and Improving Sample Complexity (ICLR 2024 · similarity 0.89)
- Offline Behavior Distillation (NeurIPS 2024 · similarity 0.89)
- Q-Supervised Contrastive Representation: A State Decoupling Framework for Safe Offline Reinforcement Learning (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.